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TRAINING  AIDE:  RESEARCH  AND 
GUIDANCE  FOR  EFFECTIVE  TRAINING 
USER  GUIDE 


Introduction  to  TARGET 


What  is  TARGET? 

TARGET  stands  for  Training  Aide:  Research  and  Guidance  for  Effective  Training 
(TARGET).  TARGET  is  a  database  of  training  effectiveness  research  findings  that  provides 
information  on  the  relationships  between  training  methods  and  perfonnance.  The  training  tool  is 
comprised  of  a  database  of  training  effectiveness  research  findings  and  a  graphical  user  interface 
(GUI).  TARGET  provides  query-based  searches  of  the  training  literature  database,  numerous 
visualizations  of  the  relationships  between  training  methods  and  performance,  and  it  is 
updateable  as  additional  training  research  is  generated  so  that  the  tool  stays  current  with  state-of- 
the-art  training  research  developments.  It  was  designed  to  assist  users  in  better  understanding  the 
relative  effectiveness  of  different  training  methods  for  acquiring/transferring  cognitive  skills 
involved  in  complex  Anny-relevant  task  domains.  The  database  includes  six  primary  types  of 
training  methods  (i.e.,  part  task  training,  task  difficulty  sequencing,  error  prevention,  scaffolding, 
learner  control,  and  exploratory  learning),  as  well  as  factors  that  moderate  their  effectiveness, 
such  as  skill  type  being  trained,  trainee  characteristics  (e.g.,  experience  level),  and  type  of 
training  performance  outcome  (e.g.,  procedural  knowledge,  transfer).  In  sum,  TARGET  is 
designed  to  assist  users  in  identifying  appropriate  training  methods  (via  underlying 
computational  algorithms)  for  different  combinations  of  tasks/skills  being  trained  (or  types  of 
trainees  that  would  participate  in  the  intended  training)  and  the  performance  outcomes  sought. 

Who  are  the  Intended  Users  of  TARGET? 

TARGET  is  geared  towards  any  user  interested  in  the  relationships  between  training  methods 
and  perfonnance;  however,  two  user  types  were  directly  considered  in  the  development  of 
TARGET. 

•  Training  Developers  -  Training  professionals  (e.g.,  instructional  system  designers) 
involved  in  the  design  and  development  of  training  programs. 

•  Training  Researchers  -  Researchers  involved  in  studying  training  effectiveness. 


1 


How  to  Think  about  TARGET? 


TARGET  use  may  be  framed  by  the  following  general  question: 

Given  a  certain  task  to  be  trained,  a  set  of  trainees,  and  training  goals  -  what  training  media 
and  methods  are  likely  to  be  most  effective? 

What  is  the  Purpose  of  this  User  Guide? 

This  user  guide  is  intended  to  provide  the  user  with  a  comprehensive  guide  for 
understanding  and  utilizing  TARGET.  It  contains  both  written  guidelines  and  associated  images. 


2 


Gaining  Access  to  TARGET 


Who  Can  Access  TARGET? 

TARGET  is  a  web  based  tool;  the  website  is  publicly  accessible  at 
http ://bldr- webtest. alionscience. com/T arget/  (  Figure  1).  Anyone  with  a  username  and  password 
can  access  TARGET.  Users  simply  to  define  their  username  and  password  and  provide  a  valid 
email  address  for  registration. 


Tramig  Aid*  N— ircft  and  Gtadanc*  tor  Eftoctov*  Training 


Home  About) 


"Given  a  certain  task  to  be  trained,  a  set  of  trainees,  and  training  goals  -  what  training  media  and  methods  are  likely  to  be  most 
effective?" 


To  answer  this  question,  TARGET  was  developed  by  Alion  Science  and  Technology  under  sponsorship  of  the  U.S.  Army  Research  Institute  (contract  W91WAW-09-C-0081). 

TARGET  can  benefit: 

•  Training  Developers 

•  Training  Researchers 

TARGET  organizes  research  findings  of  training  transfer  and  provides: 

•  Visualizations  of  Research  Findings  from  multiple  Meta-analyses 

•  Summaries  of  Key  Training  Effectiveness  Articles 

•  A  Searchable  Evidence  Base  of  the  Literature 

The  focus  is  on  military-relevant  tasks  with  complex  cognitive  skills.  TARGET  will  use  research  results  to  help  you  select  training  methods  for  different  combinations  of  tasks 
and  trainee  experience. 


Register  Log  in 


©  2013  -  Afion  Science  and  Technology 


Figure  1.  TARGET  Log-In  Page 
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Registering  as  a  User  in  TARGET 

The  register  function  is  how  you  may  register  yourself  as  a  user.  Registration  allows  you  to 
create  a  username  and  password  and  obtain  access  to  the  tool.  From  the  log  in  page  of 
TARGET: 

1)  Select  “Register”  on  the  log-in  page  (Figure  1) 

2)  Enter  a  user  name  and  password  (Figure  2) 

3)  Confinn  the  password 

4)  Enter  a  valid  email  address  (Figure  2) 

5)  Click  on  the  Register  button  (Figure  2) 


Register.  Create  a  new  account. 

Registration  Form 

User  name 

Password 
Confirm  password 

II 

Email 


|  Register  | 

Figure  2.  Register  Page 

The  only  requirements  for  registration  are  that  (1)  the  selected  username  is  unique  (i.e.,  does 
not  already  exist  for  another  TARGET  user),  and  (2)  you  provide  a  valid  email  address. 

Users  are  categorized  as  Administrator  or  Contributor.  By  default  all  new  users  are 
contributors,  meaning  that  they  can  add  new  documents,  search,  edit/delete  documents  they 
created,  and  visualize  training  effectiveness  results.  Administrators  can  delete/modify  any 
documents  and  can  change  user  categories. 

How  to  Get  More  Information  About  TARGET  and  Access  to  TARGET? 

Additional  infonnation  about  TARGET,  as  well  as  contact  infonnation  for  the  TARGET 
administrator  are  available  under  the  “About”  link  in  the  tab  menu  at  the  top  of  the  window. 
From  the  log  in  page  of  TARGET,  the  user  may  get  additional  information  by  selecting  about. 
The  about  function  allows  users  (whether  registered  or  not)  to  read  summary  information  about 
the  tool.  Also  available  under  the  “About”  link  is  contact  infonnation.  You  can  contact  either 
the  TARGET  program  manager  or  the  U.S.  ARI  sponsor  directly  through  the  provided  phone 
number  or  email  address  listed  on  the  homepage  of  TARGET. 
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Logging  Into  TARGET 


TARGET  uses  a  straightforward  sign  in  process.  Logging  in  requires  you  to  enter  your 
username  and  password,  and  then  selecting  “Tog  in”  Upon  signing  in  you  are  taken  to  the  home 
page  of  TARGET. 

Changing  Your  Password 

To  change  your  password,  click  on  your  username  at  the  top  of  any  page  preceded  by  the  text 
“Hello,”.  TARGET  will  open  the  Change  Password  page.  Enter  your  current  password,  a  new 
password  and  the  new  password  again  for  confirmation.  Click  the  “Change  Password”  button. 
TARGET  will  display  a  message  at  the  top  of  the  window  that  your  password  has  been  changed. 


Figure  3.  Change  Password 
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General  Navigation  of  TARGET 


Home  Page 

The  Home  page  is  the  top  level  page  of  TARGET  for  any  user  logged  into  the  tool 
(Figure  4).  All  other  interfaces  in  TARGET  link  back  to  the  home  page.  The  primary 
navigation  options  available  on  the  left  side  of  the  home  page  are  also  available  as  links  at  the 
top  of  the  window. 


TARGET 

Training  Aid*:  Research  and  Guidance  for  Effective  Training 


Hello,  bolott !  Leo  off 

Home  Tasks  Methods  Documents  Help  About  Admin 


Target  (Training  Aide:  Research  and  Guidance  for  Effective  Training) 


What  do  you  want  to  do? 


Explore  Tasks 


ft  V* 

►  e 


See  the  relationship  between  training  methods  and 
tasktype.  Helps  determine  which  types  of  training 
are  likely  to  work  for  the  task  to  be  trained. 


Explore  Methods 


M  m 

r 

- 

p 

See  how  different  factors  (ability,  tasktype, 
difficulty)  Impact  the  usefulness  of  a  training 
method.  Helpful  in  understanding  empirical  findings 
and  gaps. 


Explore  Documents 


Search  and  view  summaries  from  the  training 
effectiveness  literature.  Includes  individual  papers  as 
wells  as  training  method  overviews. 


Add  New  Document 


Enter  information  from  new  studies  into  the 
research  database.  Gives  you  the  ability  to  compare 
the  findings  of  your  study  with  the  information 
already  housed  in  TARGET. 


Figure  4.  TARGET  Home  Page 


Menu  Navigation 

Menu  navigation  links  are  always  present  at  the  top  of  the  window  (Figure  5).  The  menu 
navigation  is  provided  by  a  fixed  set  of  options.,  These  include:  Home,  Tasks,  Methods, 
Documents,  Help,  About,  Admin  and  Log  off  (Note:  Admin  is  available  to  Administrators 
only).  These  navigation  options  allow  you  to  quickly  jump  between  the  main  components  of 
TARGET. 
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Hello,  bplott !  Log  off 

Home  Tasks  Methods  Documents  Help  About  Admin  j 

Authors 

Data  Sources 


Figure  5.  Top  Menu 

If  you  hover  the  mouse  over  Documents,  Help,  or  the  Admin  menu  item  additional  options 
are  displayed.  Under  Document,  Authors  and  Data  Sources  are  displayed  as  sub-menus 
(Figure  5).  Under  Help  you  can  select  Definitions  and  under  Admin  you  can  optionally  select 
Categories,  Attributes,  or  Document  Summary  MetaData. 

Home 

Choose  this  option  if  you  wish  to  return  to  the  top  level  home  page  of  TARGET. 

Documents 

Choose  this  option  if  you  wish  to  link  to  the  “Explore  Documents”  component  of  TARGET. 
You  can  also  select  the  “Authors”  submenu  to  view  the  list  of  authors  in  the  database,  add  new 
authors,  and  view  documents  (e.g.,  a  research  study)  associated  with  any  author.  Selecting  the 
“Data  Sources”  submenu  allows  you  to  view  the  list  of  data  sources  in  the  database  and  add  new 
sources. 

Methods 

Choose  this  option  if  you  wish  to  link  to  the  “Explore  Methods”  component  of  TARGET. 

Tasks 

Choose  this  option  if  you  wish  to  link  to  the  “Explore  Tasks”  component  of  TARGET. 

Help 

Choose  this  option  if  you  wish  to  access  the  online  help.  The  help  menu  provides  you  with 
access  to  an  indexed  help  menu.  Help  topics  may  be  browsed  by  the  explorer  tree  or  by 
searching  on  keywords.  You  can  also  select  the  “Definitions”  submenu  to  view  detailed 
definitions  of  selected  training  methods. 
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About 


The  “About”  option  provides  background  infonnation  on  TARGET  and  provides  contact 
information. 

Logoff 

Choose  this  option  if  you  wish  to  sign  out  of  the  system.  Logging  out  returns  you  to  the  log 
in  screen. 

Admin 

The  Admin  menu  option  is  only  visible  to  users  with  Administrator  privileges.  The  Admin 
menu  offers  three  submenu  choices:  Categories,  Attributes,  or  Document  Summary  MetaData. 
The  “Categories”  option  allows  you  to  view/edit  the  categories  that  are  displayed  on  the 
Document  attributes  page.  The  “Attributes”  option  lets  you  view/edit  the  attributes  that  are 
displayed  under  each  category  on  the  Document  attributes  page.  The  “Document  Summary 
MetaData”  option  allows  you  to  view/edit  the  document  fields  and  field  definitions. 


Drop-Down  Menus 

Drop-down  menus  are  used  frequently  throughout  the  tool.  By  clicking  the  down  arrow  on 
the  right  side  of  the  menu,  a  number  of  options  are  made  available.  Options  presented  in  a  drop¬ 
down  menu  are  mutually  exclusive  -  only  one  item  may  be  selected  at  a  time. 

To  make  a  drop-down  menu  selection,  position  the  cursor  over  the  down  arrow  on  the  right 
side  of  the  menu,  move  the  cursor  over  the  desired  option  in  the  drop-down  menu  to  highlight 
your  choice,  and  then  left  click  the  mouse  to  make  your  selection  (Figure  6). 


*  Document  Type 

Select  a  Document  Type  fr 

nsna 

Book 

Book  Chapter 
Journal  Article 
Technical  Report 
Conference  Paper 
Dissertation 


r|Group  By  Moderator  a 

Moderator  Type: 

Task  Factors  !▼! 

Task  Factors 

Individual  Factors 

Training  Method  Specific 

Figure  6.  Example  Drop-Down  Menus 
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Radio  Buttons 


Radio  button  menus  are  available  throughout  the  tool.  They  allow  you  to  make  a  single 
selection  among  a  range  of  choices.  The  options  available  in  a  radio  button  menu  are  mutually 
exclusive  options;  therefore,  only  one  item  may  be  selected  at  a  time. 

To  make  a  radio  button  menu  selection  within  a  menu  set,  position  the  cursor  over  the  empty 
radio  button  to  the  left  of  the  desired  item  and  left  click  the  mouse  to  make  your  selection.  The 
radio  button  to  the  left  of  the  selected  item  will  be  filled  in  with  a  dot  to  indicate  selection 
(Figure  7).  Radio  buttons  have  a  small  “i”  icon  next  to  the  label  that  provide  short  definitions  of 
the  associated  option.  Hover  your  mouse  over  the  “i”  to  see  the  definition. 


Perceptual  p 
Psychomotor  p 
Interpersonal  p 
Cognitive-Declarative  p 
Cognitive-Procedural  p 
Cognitive-Problem  Solving  p 
Cognitive-Spatial  Reasoning  p 
Cognitive-Quantitative  Reasoning  p 
Cognitive-Complex  Decision  Making  i; 


Figure  7.  Radio  Button  Example 


Check  Boxes 

Check  box  menus  allow  you  to  make  a  number  of  selections  among  a  range  of  choices. 
Unlike  radio  button  or  drop-down  menus,  check  box  menus  are  not  mutually  exclusive;  multiple 
items  may  be  selected  at  one  time. 

To  make  a  check  box  menu  selection  within  a  menu  set,  position  the  cursor  over  the  empty 
check  box  to  the  left  of  the  desired  item  and  left  click  the  mouse  to  make  your  selection.  The 
check  box  to  the  left  of  the  selected  item  will  be  filled  with  a  check  mark  to  indicate  selection 
(Figure  8). 


Individual  Factors 
Ability  ■ 

Agey 

Education  Level  r 
Experience 1 1 
Motivation 

Figure  8.  Checkbox  Example 
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Free  Text  Fields 


A  number  of  free  text  fields  are  available  in  TARGET.  The  free  text  fields  allow  you  to  type 
infonnation.  For  example,  the  Document  Title  field  under  Create  New  document  allows  you  to 
type  the  document  title  in  the  text  field.  (Figure  9). 


Create 

hw..  a: 

*  Document  Title 

A 

Figure  9.  Free  Text  Field  Example 


Left/Right  Mouse  Navigation 

Throughout  TARGET,  you  can  navigate  the  tool  by  utilizing  left  or  right  mouse  clicks.  Left 
mouse  clicks  are  used  to  make  selections  (e.g.,  radio  buttons,  check  boxes,  and  navigation  links). 
Right  mouse  clicks  are  used  to  reveal  embedded  options.  For  example,  right  mouse  clicks  will 
open  the  hidden  options  when  interacting  with  visualizations. 


Embedded  Links  Between  Components  of  TARGET 

Throughout  TARGET,  each  component  (i.e.,  Explore  Tasks,  Explore  Methods,  Explore 
Documents,  and  Add  New  Document),  as  well  as  a  number  of  additional  features  is  set  up  to 
allow  cross  linkages  between  components  in  complementary  ways.  For  example,  within  the 
Explore  Methods  component  of  TARGET,  the  forest  plot  allows  users  to  interactively  connect  to 
the  document  summary  for  any  individual  research  study  displayed  in  the  forest  plot. 
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Figure  1.  TARGET  High  Level  Overview  with  Links  Highlighted 


The  following  is  a  complete  list  of  all  embedded  cross-component  links  available  in 
TARGET: 

•  Add  New  Document  links  to  Search  Literature  once  a  document  summary  has  been 
added. 

•  Add  New  Document  links  to  Explore  Methods  once  contrast  data  (individual  study 
result  data)  has  been  entered  and  the  associated  effect  size  statistics  are  computed  so 
you  can  visualize  the  effects  of  the  new  data. 

•  Explore  Tasks  links  to  Explore  Methods  by  interacting  with  a  training  method  node  in 
the  visualization. 

•  Explore  Tasks  links  to  Explore  Documents  by  interacting  with  a  training  method  or 
task  type  node  in  the  visualization. 

•  Explore  Methods  links  to  Document  Summaries  by  interacting  with  study  graphics  in 
the  forest  plot. 

•  Explore  Documents  links  to  Add  New  Document  by  interacting  with  the  “Create 
New”  link  from  the  document  view. 
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Choosing  What  To  Do  From  the  Home  Page 


The  home  page  is  the  top  level  page  of  TARGET  for  any  user  logged  into  the  tool 
(  Figure  1 1).  All  other  interfaces  in  TARGET  link  back  to  the  home  page.  The  home  page 
offers  links  to  each  of  the  four  main  components  of  TARGET:  Explore  Tasks,  Explore  Methods, 
Explore  Documents  and  Add  New  Document.  All  of  the  top  menu  navigation  options  are  also 
available  on  the  home  page  (as  well  as  everywhere  else  in  TARGET)  including  links  to  Tasks, 
Methods,  Documents,  Help,  About,  Admin  (available  for  administrators  only)  and  Log  off. 


TARGET 

Training  Aid*:  Research  and  Guidance  for  Effective  Training 


Hello,  bclctt !  Lea  off 


Home  Tasks  Methods  Documents  Help  About  Admin 


Target  (Training  Aide:  Research  and  Guidance  for  Effective  Training) 


What  do  you  want  to  do? 


Explore  Tasks 


See  the  relationship  between  training  methods  and 
task  type.  Helps  determine  which  types  of  training 
are  likely  to  work  for  the  task  to  be  trained. 


Explore  Methods 


r 

r 

* 

i 

See  how  different  factors  (ability,  task  type, 
diffficulty)  Impact  the  usefulness  of  a  training 
method.  Helpful  in  understanding  empirical  findings 
and  gaps. 


Explore  Documents 


Search  and  view  summaries  from  the  training 
effectiveness  literature.  Includes  individual  papers  as 
wells  as  training  method  overviews. 


Add  New  Document 


Enter  information  from  new  studies  into  the 
research  database.  Gives  you  the  ability  to  compare 
the  findings  of  your  study  with  the  information 
already  housed  in  TARGET. 


Figure  11.  TARGET  Home  Page 


Explore  Tasks 

This  options  allows  you  to  visually  explore  accumulated  research  evidence  of  different 
training  methods  within  specific  task  types.  Click  the  link  to  “Explore  Tasks”  on  the  left  side  of 
the  home  page.  This  component  is  also  available  through  the  “Tasks”  menu  at  the  top  of  any 
interface  window. 


12 


Explore  Methods 


This  option  allows  you  to  visually  explore  infonnation  on  the  effectiveness  of  different 
training  methods  and  moderators  of  the  effectiveness  of  each  training  method.  Click  the  link  to 
“Explore  Methods”  on  the  left  side  of  the  home  page.  This  component  is  also  available  through 
the  “Methods”  menu  at  the  top  of  any  interface  window.  Access  to  Explore  Methods  may  also 
be  gained  through  interactive  links  in  the  Explore  Tasks  visualizations  and  Add  New  Document 
components. 


Explore  Documents 

This  option  allows  you  to  search  and  review  the  training  literature  contained  in  the  database 
of  research  findings  using  queries  built  around  document  attributes  (e.g.,  training  method  used, 
task  category,  individual  factors,  delivery  method).  Click  the  link  to  “Explore  Documents”  on 
the  home  page.  This  component  is  also  available  through  the  “Document”  menu  at  the  top  of 
any  interface  window.  Access  to  Explore  Documents  may  also  be  gained  through  interactive 
links  in  the  Explore  Tasks  visualizations,  Explore  Methods  forest  plots,  and  Add  New  Document 
components. 


Add  New  Document 

Choose  this  option  if  you  wish  to  enter  infonnation  from  new  studies  into  the  research 
database.  Click  the  link  to  “Add  New  Document”  on  the  home  page.  Access  to  the  Add  New 
Document  function  may  also  be  gained  by  selecting  the  “Create  New”  link  under  and  Explore 
Documents  components. 
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Explore  Tasks 


Visually  explore  accumulated  research  evidence  of  different  training  methods  within  specific 
task  types.  The  task  types  included  in  TARGET  are  shown  below: 

•  Perceptual :  Skills  involved  in  detecting  and  interpreting  sensory  information  in  order  to  gain 
awareness  and  support  perfonnance.  The  ability  to  detect  and  use  sensory  cues.  Include 
some  perceptual  reasoning  tasks. 

•  Psychomotor.  Physical  skills  -  movement,  coordination,  manipulation,  dexterity,  strength, 
and  speed;  includes  both  fine  motor  skills  and  gross  motor  skills. 

•  Interpersonal :  Social  interaction  skills,  such  as:  communicate,  exchange  infonnation, 
persuade,  build  and  maintain  relationships,  manage  conflict,  and  interact  effectively. 

•  Cognitive-Declarative :  This  category  includes  all  verbal  knowledge  categories  including 
facts,  principles,  knowledge  organization,  and  verbal  or  graphical  mental  models,  concept 
maps,  etc. 

•  Cognitive-Procedural :  This  category  is  meant  to  include  performance  of  basic  procedural 
skills,  where  the  procedure  may  include  cognitive  steps  and  basic  perceptual  and  motor  steps 
such  as  those  involved  in  navigating  a  computer  interface.  Constrained  sequences  of 
physical  and  cognitive  activities  perfonned  in  predictable  situations. 

•  Cognitive-Problem  Solving :  This  category  involves  application  of  principles,  rules  and 
concepts  to  process  information  and  solve  problems.  Includes  general  logical  reasoning 
skills,  such  as  inductive,  deductive,  diagnostic  skills,  etc. 

•  Cognitive-Spatial  Reasoning :  Visual-spatial  skills  -  skill  in  representing,  transfonning, 
generating,  and  recalling  symbolic,  nonlinguistic  information.  Includes:  mental  rotation; 
spatial  perception  -  ability  to  determine  spatial  relationships  with  respect  to  own  body 
orientation;  spatial  visualization  -  multi-step  manipulations  of  spatially  presented 
information,  requiring  analysis  of  the  relationship  between  different  spatial  representations. 

•  Cognitive-Quantitative  Reasoning :  Application  of  mathematical  concepts  and  skills  to  solve 
problems. 

•  Cognitive-Complex  Decision  Making :  This  category  involves  situation  assessment  and 
decision  making  in  complex,  dynamic  and  time-sensitive  environments  with  changing 
situations,  attentional  demands,  application  of  strategies,  multiple  goals,  and  may  involve 
coordination  of  perceptual  and  motor  skills. 

Overview  of  Explore  Tasks 

The  Explore  Tasks  component  of  TARGET  provides  you  with  a  range  of  interactive  features 
for  viewing  accumulated  evidence  by  task  type  in  order  to  identify  which  training  methods  are 
likely  to  be  useful  in  training  a  particular  task  type  of  set  of  task  types  (Figure  12).  The 
component  provides  you  with  a  range  of  interactive  features  for  viewing  accumulated  evidence 
by  task  type  in  order  to  identify  which  training  methods  are  likely  to  be  useful  in  training  a 
particular  task  type  or  set  of  task  types.  The  Explore  Tasks  component  offers  four  functions:  (1) 
View  a  single  task,  (2)  Compare  two  tasks,  (3)  Link  to  Explore  Methods,  and  (4)  Link  to  Explore 
Documents. 


14 


Task  -  Method  Exploration 

|N  Tl - 


©  Perceptual  y 
O  Psychomotor  y 
O  Interpersonal  y 
O  Cognitive-Declarative  y 
O  Cognitive-Procedural  y 
O  Cognitive-Problem  Solving  y 
O  Cognitive-Spatial  Reasoning  y 
O  Cognitive-Quantitative  Reasoning  y 
O  Cognitive-Complex  Decision  Making  y 


Compare  Task  - 


©None  y 
Perceptual  y 
O  Psychomotor  y 
O  Interpersonal  y 
O  Cognitive-Declarative  y 
O  Cognitive-Procedural  y 
O  Cognitive-Problem  Solving  y 
O  Cognitive-Spatial  Reasoning  y 
O  Cognitive-Quantitative  Reasoning  y 
O  Cognitive-Complex  Decision  Making  y 
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Figure  12.  Explore  Tasks  Page 


Explore  Tasks  -  Key 

To  view  the  key  for  the  Explore  Tasks  page,  click  on  the  up  arrow  to  the  right  of  the  “Key” 
word,  /\  .  TARGET  will  expand  the  Key  area  as  shown  below. 


Key  ▲ 


Controls  - 

Right-dick  on  a  node  for  link  options 
Hover  on  a  node  for  more  details 

Press  Control  and  use  the  mouse  scroll  wheel  to  zoom  in  and  out 


Lines 

Green  line 

Evidence  of  benefit  <  j) 

Red  line 

Evidence  of  cost  <  j) 

Black  line 

Evidence  of  neutral  ij) 

Dotted  tine 

Inconclusive:  limited  data  <j) 

No  line 

Extrapolated  <j) 

Line  thickness  reflects  amount  of  data  D 

Rings 

Dark  Green  ring 

Strong  evidence  of  benefit 

Green  ring 

Evidence  of  benefit  <j) 

Yellow  ring 

Neutral/Inconclusive  <j) 

Red  ring 

Evidence  of  cost  <j) 

The  closer  the  training  method  is  to  the  center, 
the  stronger  the  evidence  that  it  benefits  training 

Figure  13.  Explore  Tasks  Key 


15 


View  a  Single  Task 


The  view  a  single  task  function  allows  you  to  select  a  task  to  view  the  accumulated  research 
evidence  of  different  training  methods.  It  is  important  to  note  that  the  evidence  displayed 
reflects  only  on  the  information  contained  in  the  TARGET  database.  To  view  a  single  task, 
select  one  radio  button  for  the  desired  task  from  the  “Primary  Task”  options  (Figure  14).  To 
view  the  definition  of  any  of  the  tasks  hover  your  mouse  over  the  “i”  button  to  the  right  of  the 
task  name. 


-  Primary  Task  - 

®  Perceptual  ■. y 
©  Psychomotor  y 
©Interpersonal  y 
O  Cognitive-Declarative  y 
©  Cognitive-Procedural  y 
©Cognitive-Problem  Solving  y 
©Cognitive-Spatial  Reasoning  y 
©  Cognitive-Quantitative  Reasoning  y 
'©Cognitive-Complex  Decision  Making  -,y 


Figure  14.  Primary  Task  Options 


Upon  selecting  a  task,  the  tool  graphically  displays  strength  of  evidence  for  cost/benefit  on 
all  six  training  methods  for  the  selected  task.  The  task  exploration  graphs  show  a  “bull’s-eye” 
with  the  task  type  in  the  middle  of  the  “bull’s-eye.”  The  training  strategies  surround  the  task 
type.  The  closer  the  strategy  to  the  center,  the  stronger  the  evidence  that  the  strategy  benefits 
training  transfer  performance.  A  vote  count  is  used  to  calculate  strength  of  support.  The  vote 
count  is  the  percentage  of  studies  related  to  that  task  type  that  showed  a  benefit  of  the  training 
strategy  being  analyzed.  For  example  if  there  were  10  studies  of  procedural  tasks  using  the 
worked  example  training  strategy  and  7  of  them  showed  a  benefit  of  worked  examples,  the  vote 
count  is  70%.  The  goal  of  this  feature  is  to  provide  you  with  “rule  of  thumb”  data  regarding  the 
relative  usefulness  of  different  strategies. 

A  “hover  over”  feature  displays  a  pop  up  window  displaying  a  ratio  of  the  number  of 
contrasts  that  show  a  training  benefit  to  the  total  number  of  contrasts  for  that  training  method. 
This  information  is  displayed  in  a  popup  window  when  the  cursor  is  placed  on  top  of  a  training 
method  node  in  the  graph  (Figure  15).  In  TARGET,  a  contrast  is  considered  a  comparison  of 
treatment  group  versus  control  group,  primarily  defined  for  the  purpose  of  creating  an  effect  size 
estimate  for  the  magnitude  of  the  difference  in  transfer  perfonnance  between  the  two  groups. 
Documents  may  have  multiple  contrasts  when  there  are  a  number  of  different  training  treatment 
groups  and/or  individual  difference  measures. 
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Figure  15.  Bubble  Graph  with  Hover  Example 


The  closer  the  training  method  is  to  the  center,  the  stronger  the  evidence  that  that  method 
benefits  training  transfer. 

The  ring  color  and  distance  from  the  center  indicate: 

•  Dark  Green  Ring  -  Strong  evidence  of  benefit 

•  Green  Ring  -  Evidence  of  benefit 

•  Y ellow  Ring  -  Neutral/Inconclusive 

•  Pink  Ring  -  Evidence  of  Cost 

The  link  type  and  color  indicate: 

•  Green  Line  -  Evidence  of  Benefit 

•  Red  Line  -  Evidence  of  Cost 

•  Black  Line  -  Neutral 

•  Dotted  Line  -  Inconclusive 

•  No  Line  -  Extrapolated 


If  the  Vote  Count  was  greater  than  56%,  it  was  classified  as  benefit.  If  the  Vote  Count  was 
between  44%  and  45%  it  was  classified  as  neutral.  If  the  Vote  Count  was  less  than  44%,  it  was 
classified  as  cost.  A  training  strategy  was  classified  as  a  strong  benefit  if  the  number  of  studies 
was  6  or  greater  AND  the  meta-analysis  showed  a  significant  benefit.  Strong  benefit,  benefit,  and 
cost  are  connected  with  a  regular  line  as  long  as  there  were  at  least  three  studies  contributing  to 
the  vote  count.  If  there  were  less  than  three  studies,  the  line  is  dashed.  If  there  was  no  data,  the 
there  is  no  connecting  line. 

Compare  Two  Tasks 

The  compare  tasks  function  allows  a  user  to  select  a  second  task  to  view  in  conjunction  with 
the  first.  To  compare  findings  for  two  tasks,  select  one  radio  button  for  the  desired  comparison 
task  from  the  “Compare  Task”  options  (Figure  16).  To  view  the  definition  of  any  of  the  tasks 
hover  your  mouse  over  the  “i”  button  to  the  right  of  the  task  name. 
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■  Compare  Task  - 

©None  y 
Perceptual  y 
O  Psychomotor  >y 
'•  Interpersonal 
©Cognitive-Declarative  y 
Cognitive-Procedural  i 
Cognitive-Problem  Solving  *y 
Cognitive-Spatial  Reasoning  y 
©Cognitive-Quantitative  Reasoning  ,y 
©Cognitive-Complex  Decision  Making  y 


Figure  16.  Compare  Task  Selection 

Upon  selecting  the  second  task,  the  tool  graphically  compares  strength  of  evidence  for 
cost/benefit  on  all  six  training  methods  for  the  selected  tasks.  The  “hover  over”  feature  works 
the  same  as  before  (Figure  17). 


uplorstoryN 
Learning  “J 

/  \  Exploratory  Learning 

/  \  Psychomotor:  1/3  Studies  showed  benefit 

/  1  Cognitive-Problem  Solving:  3/17  Studies  showed  benefit 


Figure  17.  Hover  Example  for  Two  Task  Types 
Link  to  Explore  Methods 

The  link  to  the  Explore  Methods  component  allows  the  user  to  interactively  link  to  the  forest 
plots  under  the  Explore  Methods  component.  To  link  to  the  Explore  Methods  component,  right 
click  the  training  method  node,  and  then  select  “View  Effects  Data”  by  left  clicking  the  link 
(Figure  18).  By  interacting  with  the  graphical  display  of  a  training  method,  the  user  is  able  to 
link  to  and  view  a  forest  plot  of  the  selected  training  method. 


Browse  Studies 

7T 

/ 

■ 

l 

i 

i 

View  Effects  Data 

Figure  18.  Right  Click  Menu 
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Link  to  Explore  Documents 

The  Explore  Documents  component  allows  the  user  to  interactively  link  to  the  document 
summaries  under  the  Explore  Documents  component.  To  link  to  the  Explore  Documents 
component,  right  click  any  training  method  node  or  the  task  type  node,  and  then  select  “Browse 
Studies”  by  left  clicking  the  link  (Figure  18).  By  interacting  with  the  graphical  display  of  a 
training  method  or  task  type,  the  user  is  able  to  link  to  and  view  document  summaries  for  the 
selected  node. 

Zooming  the  Bubble  Chart  In/Out 

You  can  change  the  zoom  level  of  the  bubble  chart  visualization  by  clicking  the  mouse  in  the 
bubble-chart  area  and  then  pressing  the  +  (plus)  key  to  zoom  in  or  the  -  (minus)  key  to  zoom  out. 
Alternatively,  you  can  click  in  the  bubble  chart  area  then  hold  the  Shift  and  Control  keys  down 
while  scrolling  the  middle  mouse  wheel  up  to  zoom  in  or  scroll  down  to  zoom  out. 

Panning  the  Bubble  Chart 

You  can  pan  the  bubble  chart  view  using  the  scroll  bars.  Alternatively,  you  can  click  in  the 
bubble-chart  area  then  hold  the  Shift  key  down  while  moving  the  middle  mouse  wheel  to  pan 
horizontally  or  hold  the  Alt  key  down  while  moving  the  middle  mouse  wheel  to  pan  vertically. 
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Explore  Methods 


This  section  of  TARGET  allows  you  to  visually  explore  effect  size  infonnation  on  the 
effectiveness  of  different  training  methods  and  moderators  of  the  effectiveness  of  each  training 
method. 

Overview  of  Explore  Methods 

The  Explore  Methods  component  of  TARGET  (Figure  19)  provides  you  with  interactive 
forest  plots  of  effect  size  statistics  for  a  range  of  studies  under  one  of  six  training  methods:  Part 
Task  Training,  Increasing  Difficulty,  Learner  Control,  Exploratory  Learning,  Error  Prevention, 
and  Scaffolding.  The  Explore  Methods  function  offers  eleven  primary  functions:  (1)  View  a 
training  method,  (2)  Explore  moderators,  (3)  View  by  outcome,  (4)  Expand/collapse  the 
summary  effects,  (5)  Fixed/random  model  toggle,  (7)  Link  to  single  document  summary,  (8) 
Effect  size  extrapolation  function  toggle,  (9)  Include  new  studies  function  toggle,  (10)  Include 
studies  with  no  outcome  measures  toggle,  and  (11)  Customize  the  forest  plot  layout. 


Select  Method:  Error  Prevention  0 

Group  and  Filter  A 
Group  By  Moderator  ▼ 

Included  Outcome  Measures  ▼  Advanced  Options  ▼ 

Apply  Changes  | 

Edit  Layout 


Error  Prevention 
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Figure  19.  Explore  Methods  Page 
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A  forest  plot  is  a  display  that  presents  a  number  of  means  and  their  confidence  intervals  in  a 
graphic  manner,  so  that  they  can  be  easily  reviewed  and  compared.  The  forest  plot  is  essentially 
a  number  of  bars  with  a  square  in  the  middle.  The  bars  represent  the  confidence  interval  and  the 
square  the  hedges’s  g  value.  The  central  vertical  line  represents  no  effect.  Squares  that  appear  to 
the  left  of  the  vertical  line  indicate  “cost”  and  squares  to  right  of  the  vertical  line  indicate 
“benefit.”  The  red  diamond  above  the  studies  represents  the  overall  effect.  For  more  details  on 
forest  plots  see  Hedges  and  Olkin  (1985). 

View  a  Training  Method 

The  first  step  for  viewing  a  forest  plot  is  selection  of  a  training  method.  The  Explore 
Methods  forest  plots  are  anchored  on  six  training  methods:  Part  Task  Training,  Increasing 
Difficulty,  Learner  Control,  Exploratory  Learning,  Error  Prevention,  and  Scaffolding.  To  select 
a  training  method  for  the  forest  plot,  highlight  and  left  click  the  method  from  the  “Select 
Method”  drop-down  menu  at  the  top  of  the  window  (Figure  20). 

Short  descriptions  of  each  method  are  given  below. 

o  Part  task  training :  Training  method  that  decomposes  complex  tasks  into  a 
series  of  smaller  tasks,  each  of  which  is  demonstrated  and  practiced  separately 
before  being  practiced  as  a  whole  task. 

o  Increasing  difficulty.  Parameters  of  the  whole  task  are  initially  set  to  lower 
levels,  to  reduce  the  intrinsic  load  during  early  training  trials,  and  then 
increased  as  training  progresses,  until  the  difficulty  reaches  the  level  of  the 
whole  task.  The  difficulty  levels  can  increase  in  either  a  fixed  or  adaptive 
schedule. 

o  Learner  control :  An  instructional  method  that  provides  trainees  with  decision¬ 
making  control  over  specific  dimensions  or  activities  within  a  structured 
learning  environment. 

o  Exploratory  learning :  A  trainee  explores  a  task  environment  on  his/her  own. 
The  level  or  type  of  guidance  given  to  the  trainee  vary  (i.e.,  only  a  user 
manual  vs.  conceptual  input  when  trainer  queried). 

o  Error  Prevention:  An  error  prevention  strategy  geared  towards  reducing  the 
complexity  of  the  transfer  task  during  initial  learning  for  the  purpose  of 
reducing  training  task  errors,  and  to  help  trainees  acquire  the  appropriate 
schema  to  assimilate  the  transfer  task  throughout  learning.  Utilizes  hard 
lockouts  and  worked  examples. 

o  Scaffolding:  Assistive  supports  are  provided  to  the  trainee  to  ease  the  demands 
of  independent  task  performance.  The  scaffold  supports  are  incrementally 
faded  out  over  time  until  the  trainee  is  executing  the  whole  task  independently 
with  minimal  demand. 
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Figure  20.  Select  Method  Drop-Down 

Upon  selection,  a  forest  plot  graphically  displays  effect  size  data  for  all  experimental  studies 
in  the  database  using  the  selected  training  method  (Figure  21).  The  selected  training  method  will 
show  up  as  the  heading  on  window.  The  forest  plot  will  display,  for  the  selected  training 
method,  individual  study  effects  as  well  as  a  summary  effect,  shown  with  a  red  diamond, 
representing  g+  for  the  full  set  of  studies  shown;  where,  g+  is  equivalent  to  the  mean  effect  size 
of  all  Hedges’  g  values  displayed. 

Hedges’  g  was  chosen  as  the  effect  size  statistic  for  three  primary  reasons.  First,  as  an  index  from  the  family  of 
standardized  mean  differences,  the  standardized  scale  of  Hedges’  g  allows  comparison  of  the  magnitude  of  the 
difference  between  groups  across  studies.  Second,  Hedges’  g  is  an  effect  size  index  commonly  associated  with 
analysis  of  variance  designs  thus  capturing  the  overlap  between  distributions  of  experimental  groups  as  the 
standardized  mean  difference.  The  majority  of  the  research  designs  reviewed  meta-analytically  for  training 
effectiveness  studies  utilized  experimental  or  quasi-experimental  research  designs.  Third,  Cohen’s  d’  index,  a 
common  index  of  standardized  mean  difference,  tends  to  be  an  upwardly  biased  estimate  of  effect  size  for  small 
samples.  Hedges’  g  is  a  sample  adjusted  correction  to  Cohen’s  d’  to  address  the  upward  bias  of  d’  (Borenstein,  et  al, 
2009;  Hedges  &  Olkin,  1985;  Sheskin,  2007). 


Figure  21.  Forest  Plot  Example 
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By  default,  individual  study  effect  g  and  standard  error  (SE)  data  is  shown  to  the  right  of  the 
study  name.  To  view  the  overall  effect  data  hover  your  mouse  over  the  red  diamond  (Figure  22). 


Figure  22.  Overall  Effects  Data  Hover 
Explore  Moderators 

The  explore  moderators  function  allows  the  user  to  filter  the  research  studies  displayed  on  a 
specific  moderator  variable.  Moderator  variables  are  mutually  exclusive  options,  and 
consequently  can  only  be  selected  one  at  a  time.  The  moderators  available  include  task  factors, 
individual  factors,  and  training  method  specific  factors.  The  task  factors  include  Task  Category 
and  Task  Difficulty.  The  individual  factors  include  Ability  and  Experience.  In  addition,  a  set  of 
2-3  training  method  specific  moderators  will  be  available  for  each  training  method  selected.  The 
training  method  specific  moderators  are  unique  to  the  implementation  of  each  training  method. 

To  select  a  moderator,  highlight  and  left  click  the  desired  moderator  from  the  “Moderator 
Type”  drop-down  menu  at  the  top  of  the  window  (Figure  23).  Then  select  an  individual 
moderator  from  the  “Moderator”  drop-down  menu.  Once  a  moderator  has  been  selected  from 
the  drop-down  menu,  click  the  “Apply  Changes”  button.  All  studies  shown  in  the  forest  plot  will 
be  grouped  by  levels  of  the  selected  moderator  variable.  A  separate  summary  effect,  shown  with 
a  red  diamond,  is  displayed  for  each  level  of  the  moderator  (Figure  24). 
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Figure  24.  Forest  Plot  with  Moderator  Groupings 
View  by  Outcome 

The  view  by  “Included  Outcome  Measure”  function  allows  you  to  specify  further  subsets  of 
outcome  data  to  display  in  the  forest  plot. 

At  the  top  of  the  filter  pane  are  two  options:  1)  Check  or  Uncheck  All  and  2)  No  Outcome 
Measures.  Use  the  “Check  or  Uncheck  All”  box  to  quickly  select  or  unselect  all  outcome 
options. 

By  default  all  studies  in  the  library  database  are  included  in  the  effect  size  calculations  that 
meet  the  boxes  checked  under  the  “Included  Outcome  Measures”  filter  area.  You  can  optionally 
include  studies  that  do  not  have  defined  outcome  measures.  To  turn  the  inclusion  of  studies  with 
no  outcome  measures  off/on,  uncheck/check  the  check  box  menu  next  to  “No  Outcome 
Measures.” 

Three  outcome  options  are:  (a)  Transfer  test  type,  (b)  Transfer  measure  type,  and  (c)  Transfer 
distance  type  (Figure  25).  You  can  hover  your  mouse  over  the  “i”  to  the  right  of  the  outcome 
options  to  view  definitions. 
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Figure  25.  Outcome  Measures  Filter  Pane 


Two  options  are  available  for  Transfer  Test  type:  Perfonnance  Tests  and  Knowledge  Tests. 
Performance  tests  capture  the  ability  of  a  trainee  to  physically  performance  a  trained  skill. 
Knowledge  tests  capture  the  ability  of  a  trainee  to  recall  information  or  problem  solve  from  a 
trained  skill. 

Two  options  are  available  for  Transfer  Measure  type:  Accuracy  measures  and  Time 
measures.  Accuracy  measures  capture  features  of  performance  related  to  correctly  perfonning  a 
transfer  task.  Time  measures  capture  features  of  performance  related  to  the  time  to  respond, 
perfonn  an  action,  or  complete  a  task. 

Three  options  are  available  for  Transfer  Distance  type:  Identical  Tests,  Near  Transfer  Tests, 
and  Far  Transfer  tests.  Identical  tests  examine  the  application  of  what  was  learned  in  training 
using  a  task  or  problem  identical  to  that  used  during  training.  Near  transfer  tests  examine  the 
application  of  what  was  learned  in  training  using  a  task  or  problem  somewhat  different,  but 
similar,  to  that  used  during  training.  Far  transfer  tests  examine  the  application  of  what  was 
learned  in  training  using  a  testing  situation  that  is  different  or  new  from  the  task  or  problem  that 
was  used  in  training. 

Outcome  filtering  is  not  mutually  exclusive,  so  any  number  of  options  within  each  of  the 
three  outcome  types  may  be  selected  at  one  time.  To  select  an  outcome  type,  check  the  box  next 
to  the  outcome  measures  to  filter  on.  The  forest  plot  will  refresh  after  you  select  the  “Apply 
Changes”  button. 

Expand/Collapse  the  Summary  Effects 

The  expand/collapse  function  allows  you  to  interact  with  the  forest  plot  display  in  order  to 
expand  the  view  to  include  effect  size  data  for  all  studies  or  collapse  the  view  to  include  only  the 
summary  effects.  A  fully  expanded  view  showing  all  studies  is  the  default  view. 

To  collapse  all  studies  and  moderator  summary  effects,  left  click  the  grey  rectangle 
displaying  a  minus  sign  '  to  the  left  of  the  “Overall”  label.  The  overall  summary  effect  will  be 
the  only  effect  size  information  displayed  as  shown  in  Figure  26. 
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Figure  26.  Expanded  and  Collapsed  Summary  Effects 


To  expand  the  overall  summary  effect  and  view  all  studies  for  the  selected  training  method, 
left  click  the  grey  rectangle  displaying  a  plus  sign  +  to  the  left  of  the  “Overall”  label. 

Similarly,  the  levels  of  a  moderator  variable  may  be  collapsed  by  left  clicking  the  grey 
rectangle  displaying  a  minus  sign  to  the  left  of  the  moderator  label  when  all  studies  for  that 
moderator  level  are  displayed.  To  expand,  left  click  the  grey  rectangle  displaying  a  plus  sign  to 
the  left  of  the  moderator  label  when  all  studies  for  that  moderator  level  are  hidden. 


Link  to  Single  Document  Summary 

The  link  to  a  single  document  summary  under  the  Explore  Documents  component  allows  you 
to  interactively  link  to  the  details  of  single  research  study  by  interacting  with  the  individual  study 
graphics.  To  link  to  a  single  document  summary  under  the  Explore  Documents  component,  left 
click  the  specific  study  name  in  the  forest  plot  (Figure  27). 
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Figure  27.  Single  Document  Link 
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Once  selected,  you  will  switch  views  to  the  document  details  function  under  the  Explore 
Documents  component  of  TARGET  showing  the  document  details  of  the  specific  study  selected 
(Figure  28). 
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Figure  28.  Document  Details  Page 
Advanced  Options 

TARGET  provides  several  advanced  filtering  options.  To  view  the  available  options 
(Figure  29),  click  the  down  arrow  next  to  the  “Advanced  Options”  text.  Each  of  the  options  is 
described  below. 
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Figure  29.  Advanced  Options  -  Include  New  Studies 

You  have  the  option  of  viewing  effect  size  data  for  just  library  data  or  for  library  data  plus 
any  new  research  studies  that  you  have  entered.  The  “Include  New  Studies”  option  allows  you 
to  toggle  on  or  off  the  inclusion  of  your  new  studies  (studies  that  have  not  yet  been  approved  by 
an  administrator). 
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To  turn  the  new  study  inclusion  off/on,  uncheck/check  the  check  box  menu  next  to  “Include 
New  Studies”  in  the  “Advanced  Options”  filter  area  (Figure  29).  This  option  only  impacts  new 
documents/contrasts  added  by  you. 

Include  Extrapolated  Data 

The  “Include  Extrapolated  Data”  option  allows  you  to  toggle  on  or  off  the  utilization  of  an 
extrapolation  algorithm  in  the  computation  of  effect  sizes  for  combinations  of  variables  that  do 
not  have  supporting  effect  size  data  in  the  tool.  For  a  more  detailed  discussion  of  the 
extrapolation  algorithm  see  Hutchins,  Carolan,  Plott,  McDermott,  &  Orvis  (2014). 

The  extrapolation  algorithm  utilizes  a  parent  effect  process  by  leveraging  higher  level  effects 
in  data  sets  where  the  combinations  are  available  to  approximate  an  effect  size. 

TARGET  can  use  an  effect  size  extrapolation  process  to  estimate  effect  sizes  for  two  types  of 
moderator  combination  subsets.  The  first  type  involves  estimating  an  effect  size  for  outcome 
variable  subsets  (i.e.,  all  contrasts  that  combine  performance  tests,  accuracy  measures,  and  near 
transfer)  that  are  not  represented  within  the  training  method  contrast  data  set,  based  on  the 
missing  attribute  (e.g.,  near  transfer)  “parent”  effect  size  value  (i.e.  performance  tests,  accuracy 
measures,  all  transfer  distances).  The  second  type  of  subset  combination  in  TARGET  includes 
one  of  the  other  moderators  (Skill  Type,  Task  Difficulty,  Ability,  Experience,  or  Training 
Method  Specific  moderators)  with  one  of  the  12  combinations  of  three  outcome  moderator 
variables. 

Two  heuristics  restrict  the  extrapolation  range.  First  all  extrapolation  is  within  the 
training  method  (e.g.,  scaffolding)  data  set.  Second,  there  must  be  at  least  one  data  point  in  a  set 
of  training  method  contrasts  for  a  moderator  variable  (e.g.,  Scaffolding:  Transfer  Distance:  Near 
Transfer)  to  be  used  as  an  extrapolation  source.  Therefore  moderator  variables  with  no  study 
contrast  data  (e.g.,  Scaffolding:  Transfer  Distance:  Far  Transfer)  are  excluded  from  estimation  by 
extrapolation  in  TARGET.  (In  these  cases  the  parent  moderator  (e.g.,  Scaffolding:  Transfer 
Distance),  or  with  a  theoretical  basis,  a  sibling  moderator  variable  (e.g.,  Scaffolding:  Transfer 
Distance:  Near  Transfer)  is  the  best  available  effect  estimate  within  the  training  method  data  set.) 


You  can  only  select  “Include  Extrapolated  Data”  if  the  conditions  are  met  -  one  choice  from 
each  Measure  (Transfer  Test,  Transfer  Measure,  Transfer  Distance).  Extrapolated  data  is 
indicated  by  an  orange  diamond  (Figure  30).  Hover  over  the  Orange  diamond  -  that  will  give 
you  g  and  the  upper  and  lower  bounds  of  the  Confidence  Interval  from  the  extrapolation. 
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Figure  30.  Extrapolated  Data 

To  turn  the  extrapolation  algorithm  off/on,  uncheck  the  check  box  menu  next  to  “Include 
Extrapolated  Data”  in  the  “Advanced  Options”  filter  area  (Figure  29). 

Fixed  Effect  or  Random  Effects  Model  Toggle 

TARGET  dynamically  calculates  overall  or  combined  effect  from  a  variety  of  studies  so  that 
you  can  see  if  the  literature  shows  that  a  particular  training  method  shows  a  cost,  a  benefit,  or  is 
inconclusive.  If  all  studies  were  equally  precise  you  could  simply  compute  the  mean  of  the 
individual  effect  sizes.  However,  typically  some  studies  are  more  precise  than  others  so  we  want 
to  assign  more  weight  to  the  studies  that  carry  more  information.  There  are  two  models  that  can 
be  used  to  assign  the  weights:  a  fixed  effects  model  and  a  random  effects  model.  The  two  make 
different  assumptions  about  the  nature  of  the  studies,  and  these  assumptions  lead  to  different 
calculations  of  the  overall  effect. 

Fixed  effect  models  assume  a  single  source  of  variation  between  effect  sizes  due  only  to 
sampling  error  (i.e.,  within-study  variance).  The  assumption  of  a  fixed  effect  model  is  that  each 
contrast’s  effect  size  is  a  sample  from  a  distribution  of  effect  sizes  around  a  true  population 
mean.  Consequently,  the  only  source  of  variation  around  the  true  population  mean  is  random 
variation  of  each  contrast’s  estimate  from  the  true  mean.  A  fixed  effect  model  is  typically 
chosen  when  the  modeler  assumes  that  each  study  is  an  equivalent  context  of  training 
assessment;  that  is,  each  experimental  study  is  assumed  to  be  estimating  the  same  true  population 
effect  rather  than  different  population  effects  associated  with  between  study  differences  in 
training  environment,  sample  characteristics,  training  tasks,  etc.  The  single  variation  parameter 
associated  with  the  assumption  a  single  true  population  mean  and  random  variation  of  each 
contrast’s  estimate  around  that  mean  is  used  to  compute  the  set  of  fixed  effect  quantities  that  are 
later  used  to  compute  the  fixed  effect  model  summary  statistics  (Borenstein,  Hedges,  Higgins  & 
Rothstein,  2009). 

In  contrast  to  the  fixed  effect  model,  the  assumption  of  a  random  effects  model  is  that  each 
contrast’s  effect  size  is  a  sample  from  a  distribution  of  effect  sizes  around  a  unique  population 
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mean.  Consequently,  two  sources  of  variation  exist  for  the  random  effects  model:  random 
variation  of  the  sample  from  the  distribution  around  the  unique  population  mean,  and  random 
variation  of  the  sample  from  the  mean  of  the  distribution  of  unique  population  means. 

A  random  effects  model  is  typically  chosen  when  the  modeler  assumes  that  each  study 
represents  a  different  context  of  training  assessment;  that  is,  each  experimental  study  is  assumed 
to  be  estimating  a  different  population  mean  due  to  between  study  differences  in  training 
environment,  sample  characteristics,  training  tasks,  etc.  The  random  effects  model  summary 
statistic  is  a  calculation  of  the  mean  of  the  distribution  of  different  unique  population  parameters. 
Therefore,  random  effects  models  assume  two  sources  of  variation,  within-study  variance  (as  in 
the  fixed  effect  model)  and  variation  due  to  true  differences  between  effect  size  estimates  (i.e., 
between-studies  variance). 

The  fixed/random  model  toggle  function  allows  you  to  switch  between  a  fixed  effect  and 
random  effects  models  for  the  computation  of  summary  statistics.  A  fixed  effect  model  is  the 
default  model  in  TARGET. 

To  select  a  random  effects  model,  check  the  box  to  the  left  of  “Use  Random  Effects  Model” 
under  the  “Advanced  Options”  filter  area  (Figure  31)  and  click  the  “Apply  Changes”  button.  To 
go  back  to  a  fixed  effect  model,  simply  uncheck  the  box  and  click  the  “Apply  Changes”  button. 
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Figure  31.  Advanced  Options 

Whenever  the  random  effects  model  check  box  is  checked  or  unchecked  and  the  “Apply 
Changes”  button  is  clicked,  all  summary  effects,  both  overall  and  for  each  moderator  variable, 
will  be  recomputed  to  reflect  the  selected  model. 

Edit  Layout 

You  can  change  the  way  the  forest  graph  is  displayed  using  the  Edit  Layout  functionality.  To 
view/edit  the  layout,  click  on  the  “Edit  Layout”  button.  TARGET  will  display  a  frame  similar  to 
Figure  32.  From  here  you  can  choose  what  information  to  display  and  set  the  width  used  for 
each  piece  of  information.  To  have  display  a  category  of  infonnation,  check  the  box  to  the  right 
of  “Visible:”  -  similarly  to  not  show  something  uncheck  the  “Visible:”  box.  To  set  the  width  of 
each  column  you  can  either  enter  a  number  in  the  “Width:”  field  or  use  the  up/down  arrows  to 
increase/decrease  the  width.  Once  you  have  made  all  the  desired  changed  to  the  layout,  click  the 
“Accept  Changes”  button  at  the  top  of  the  frame. 
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Edit  Layout 
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Figure  32.  Edit  Layout  Button  and  Options 

The  available  fields  are: 

•  Study:  Contrast  short  name. 

•  Contrast:  Short  contrast  description. 

•  ID:  Database  ID. 

•  g:  Hedge’s  g 

•  SE:  Standard  Error 

•  Cl:  95%  Confidence  Interval 

•  ForestPlot:  Plot  of  effect  data  in  a  graphical  display 

•  WeightBar:  Graphical  display  of  relative  weight  of  contrast  data 

•  Weight:  Relative  weight  of  study  contrast 

To  disregard  any  layout  changes  click  the  “Cancel”  button. 
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Explore  Documents 


Search  and  review  the  training  literature  contained  in  the  database  of  research  findings  with 
queries  built  around  document  attributes  (e.g.,  Training  Method,  Author  Name,  Year  Published, 
Document  Type,  Document  Title,  and  Reviewer). 

Overview  of  Explore  Documents 

The  Explore  Documents  component  of  TARGET  provides  users  with  a  searchable  database 
of  literature  summaries.  The  Explore  Documents  component  offers  four  functions:  (1)  View 
document  summaries,  (2)  Create  a  new  document,  (3)  View/Edit  Author  list,  and  (4)  View/Edit 
Data  source  list  (Figure  33).  The  documents  are  shown  by  default  with  20  documents  per  page. 
The  following  columns  are  shown:  Title,  Authors,  Year,  Document  Source,  Report  Type,  Date 
Created,  and  Created  By. 
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Home  Tasks  Methods  Documents  Help  About  Admin] 

Authors 

Data 

Sources 


Documents  Found:  513_ Create  New  ■» 


Title 

Authors 

Year 

Document  Source 

Report  Type 

Date  Created 

Created  By 

Using  synthetic  learning  environments 

Cannon-Bowers,  J.  A.;  Bowers,  C.  A.)  Sc 

2008 

Work  Group  Learning:  Unders 

Literature  Review 

7/26/2010  3:06:09  PM 

How  much  and  what  type  of  guidance  is 

Clark,  R.  E. 

2009 

Constructivist  Theory  Applied 

Literature  Review 

8/17/2010  9:48:52  AM 

A  new  look  at  learner  control:  Meta-ana 

Kraiger,  K.S  Jerden,  E. 

2007 

Where  is  the  Learning  in  Diste 

Meta  Analysis 

4/29/2010  12:00:00  AM 

Uncovering  workplace  interpersonal  skil 

Klein.  C.  R.}  DeRouin,  R.  E.j  Salas,  E. 

2006 

International  Review  of  Indus 

Descriptive 

4/4/2012  12:00:00  AM 

Training  for  efficient,  durable,  and  flexi 

J 
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2008 

Individual  and  Team  Skill  Dec 

Experimental  Data 

9/13/2010  11:33:30  AM 

Data  entry:  Awindowto  principles  of  tra 

Healy,  A.  F.:  Kole,  J.  A.;  Wohldmann,  E 

2010 

Successful  Remembering  and 

Experimental  Data 

9/7/2010  10:28:54  AM 

Feedback  augmentation  and  part-task 

Kirlik,  A.;  Fisk,  A.j  Walker,  N.;  Rothrock 

1998 

Making  Decisions  Under  Stres 

Experimental  Data 

7/21/2010  2:43:13  PM 

cvoeller  = 

Skill  learning,  enhancement  of 

Healy,  A.  F. 

2009 

Encyclopedia  of  the  Mind 

Descriptive 

8/31/2010  3:35:46  PM 

Perfecting  interpersonal  skills 

Valant,  L. 

2008 

Practical  Accountant 

Descriptive 

4/6/2012  12:00:00  AM 

Does  the  modality  principle  hold  for  dif 

Moreno,  R. 

2006 

Journal  of  Assisted  Learning 

Meta  Analysis 

5/2/2010  12:00:00  AM 

Using  standardized  clients  in  the  dassr 

Rogers,  A.;  Welch,  B. 

2009 

Journal  of  Teaching  in  Social 

Experimental  Data 

1/9/2013  8:12:32  PM 

Human  factors  simulation  in  construrtio 

Jaeger,  M.j  Adair,  D. 

2010 

European  Journal  of  Engineer 

Experimental  Data 

2/1/2013  2:50:14  PM 

Cognitive  tutors:  Lessons  learned 

Anderson,  J.  R.;  Corbett,  A.  T.;  Koedinc 

1995 

The  Journal  of  the  Learning  Si 

Literature  Review 

4/30/2010  2:40:12  PM 

Interpersonal  skills  in  the  reference  wo 

Pellack,  L.  J. 

2003 

The  Reference  Librarian 

Descriptive 

4/5/2012  12:45:57  PM 

Assertive  training  in  groups  using  video 

Galassi,  J.  P.i  Galassi,  M.  D.:  Litz,  M.  C 

1974 

Journal  of  Counseling  Psycho 

Experimental  Data 

10/18/2012  1:07:01  PM 

The  training  of  complex  task  performar 

Fabiani,  M.;  Buckley,  J.;  Gratton,  G.;  Cc 

1989 

Acta  Psychologica 

Experimental  Data 

9/8/2010  3:41:00  PM 
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Figure  33.  Explore  Documents 

You  can  change  the  number  of  items  displayed  per  page  to  10,  20,  or  30  by  selecting  the 
appropriate  choice  in  the  drop-down  control  at  the  bottom  of  the  list  (Figure  34). 

You  can  navigate  between  pages  using  the  page  controls  at  the  bottom  of  the  list  or  by 
entering  a  page  number  in  the  text  field  to  the  right  of  “Page.”  The  controls  are: 

•  Next  -  -  go  to  the  next  page 

•  Last  -  -  go  to  the  last  page 

•  Previous  -  -  go  to  the  previous  page 

•  Beginning  -  -  go  the  first  page 
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Figure  34.  Document  List  Page  Controls 
Sorting  the  Document  List 

You  can  sort  the  document  list  by  any  of  the  displayed  columns  (Title,  Authors,  Year, 
Document  Source,  Report  Type,  Date  Created,  or  Created  By).  To  sort  in  ascending  order 
simply  click  on  the  column  heading  -  TARGET  will  sort  the  list  and  display  arrows  with  the  up 

arrow  highlighted  next  to  the  column  name  .  To  sort  in  descending  order,  click  on  the  column 
heading  then  click  on  the  down  arrow  . 

Document  Filters 


The  Document  Filters  functionality  allows  you  to  view  individual  document  summaries  by 
document  attribute.  You  can  select  various  document  attributes  and  view  the  individual 
document  summaries  by  expanding  the  Document  Filters  area  at  the  top  of  the  page  by  clicking 
the  up  arrow, /\.  The  default  view  without  filters  for  the  Explore  Documents  component  is  to 
show  all  documents  in  the  database  (Figure  35). 
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Document  Source 

Report  Type 
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Created  By 

The  transfer  value  of  guided  learnir 
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7/26/2010  3:06:09  PM 

cvoeller 

How  much  and  what  type  of  guidam 

Clark,  R.  E. 
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cvoeller 
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cvoeller 
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cvoeller 
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2010 
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Experimental  Data 

2/1/2013  2:50:14  PM 

cvoeller 

Cognitive  tutors:  Lessons  learned 

Anderson,  J.  R.;  Corbett,  A.  T.;  Kof 

1995 

The  Journal  of  the  Learning 

Literature  Review 

4/30/2010  2:40:12  PM 

cvoeller 
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Pellack,  L.  J. 

2003 

The  Reference  Librarian 

Descriptive 
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cvoeller 

Assertive  training  in  groups  using  v 

Galassi,  J.  P.;  Galassi,  M.  D.;  Litz,  1“ 

1974 
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cvoeller 

The  training  of  complex  task  perfor 

Fabiani,  M.;  Buckley,  J.;  Gratton,  G 
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Figure  35.  Explore  Documents  with  Document  Filters 
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To  view  a  subset  of  document  summaries,  two  filtering  options  are  available:  keyword 
Boolean  searches  and/or  filtering  by  attributes.  To  filter  on  keywords,  expand  the  Document 
Filters  area  by  clicking  the  down  arrow.  Then  select  both  the  term  fields  and  Boolean  logic  by 
using  the  drop-down  menus,  and  then  enter  the  keywords  (Figure  36).  The  available  fields, 
Boolean  logic,  and  search  options  are  shown  in 

You  can  toggle  on/off  matching  the  keyword  case  by  clicking  the  “Match  Case”  checkbox. 
Once  your  filter  criteria  have  been  set,  click  on  the  Search  button.  TARGET  will  display  all 
documents  that  meet  the  filter  criteria. 


Figure  36.  Keyword  Search 
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Table  1.  Document  Filter  Options 


Field 

Description 

Boolean  Logic 

Search  Option(s) 

Title 

Name  of  the  document 

Contains,  Does  Not  Contain, 
Matches  Exactly 

Free  Text  (Match  Case 
option) 

Authors 

Name  of  one  or  more 
authors 

Contains,  Does  Not  Contain, 
Matches  Exactly 

Free  Text  (Match  Case 
option) 

Year 

Year  document 
published 

Is  In  Range,  Is  Not  in  Range 

From  Date  Drop-Down, 
To  Date  Drop-Down 

Document 

Type 

Area  in  which  the 
document  was  published 

Is,  Is  Not 

Drop-down  selection 
(Book,  Book  Chapter, 
Journal  Article, 

Technical  Report, 
Conference  Paper, 
Dissertation) 

Source 

Name  of  data  source 
document  is  appears  in 

Is,  Is  Not 

Drop-down  selection 

Report 

Type 

Category  of  document 
focus 

Is,  Is  Not 

Drop-down  selection 
(Experimental  Data, 

Meta  Analysis,  Training 
Evaluation,  Literature 
Review,  Descriptive) 

Reviewer 

User  name  of  person 
who  created  the 
document  record 

Contains,  Does  Not  Contain, 
Matches  Exactly 

Free  Text  (Match  Case 
option) 

Training 

Methods 

The  type  of  training 
reviewed  in  the 
document 

Include,  Do  Not  Include 

Drop-down  selection 
(Error  Prevention, 
Exploratory  Learning, 
Increasing  Difficulty, 
Learner  Control,  Part 
Task  Training, 
Scaffolding) 

You  can  filter  the  search  using  multiple  criteria  by  clicking  on  the  “Add  Row”  button  and 
specifying  the  additional  field(s)  and  search  options  and  clicking  the  Search  button.  For 
example,  in  the  figure  below  TARGET  will  search  for  all  documents  that  are  of  Training  Method 
type  “Part  Task  Training,”  were  published  between  2000  and  2013  and  are  Journal  Articles.  To 
delete  any  of  the  extra  search  criteria  click  on  the  “Remove  Row”  button  and  TARGET  will 
delete  the  associated  row. 
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Document  Filters  ▲ 


|  Training  Methods 

M  |  Include 

M  |  PartTasKTraining 

R 

Remove  Row 

Year 

| » 1 1  Is  In  Range 

FI  From:  |2000["-|  To:  2013  M 

|  Remove  Row  | 

Document  Type 

ftJQ? 

FI  |  Journal  Article 

ZB 

Remove  Row  Add  Row 

Figure  37.  Multiple  Criteria  Filter 
Document  Attribute  Filters 

To  filter  on  document  attributes,  expand  the  select  the  document  attributes  by  clicking  the 
“Filter  by  Attributes”  down  arrow.  Next,  select  attributes  by  checking  the  box  next  to  each 
attribute  of  interest.  Multiple  boxes  under  each  category  can  be  checked  simultaneously 
(Figure  38). 


niter  by  Attributes  a - 

Search  Criteria:  W  Match  Any  Selected  O  Match  All  Selected 


Training  Method:  Practice 

Task  Category 

Individual  Factors 

□Group  Practice# 

□Cognitive-Complex  Decision  Making  c# 

□Ability# 

□Mental  Practice!) 

□Cognitive-Declarative  # 

□Age# 

□Overlearning# 

□Cognitive-Problem  Solving  # 

□Education  Level# 

□Practice  Spacing  # 

□Cognitive-Procedural# 

□Experience# 

□Scenario-Based  Practice# 

□Cognitive-Quantitative  reasoning  # 

□Motivation# 

□Cognitive-Spatial  Reasoning# 

□Adaptive  Training'# 

.  ^Interpersonal  # 

□Perceptual# 

□Learner  Reaction# 

□Exploration# 

□Psychomotor  '# 

□Learning# 

□Increasing  Difficulty  # 

□Retention# 

□Learner  Control# 

Task  TvDe:  Military  Specific 

_ 1  DTransfer# 

□Part  Task  Training  # 

□Adaptability# 

□Transfer-Job# 

□Scaffolding# 

□Battlespace  Awareness# 

Meta-analysis 

□Training  Wheels/Error  Prevention# 

□Command  and  Control  # 

□Variable-Priority  Training# 

□Critical  Thinking# 

□Training  Difficulty# 

□Worked  Examples# 

□Digital  Skills# 

Sample 

Training  Method:  Other 

□Academic# 

□Behavior  Modeling  # 

TaskTvDe:  Interpersonal  Skills 

'  UMilitary# 

□Feedback# 

□Active  Listening  # 

□Work# 

□Prior  knowledge  activation  # 

□Conflict  Resolution  # 

Delivery  Method 

□Proceduralized  Instruction  # 

□Leadership  Skills# 

□Comparing  Delivery  Methods# 

□Team  Training# 

□Negotiation# 

□Computer-based  # 

□Reid/Real  System# 

□Relationship  Building  # 

□Gaming# 

□Simulation-based# 

TaskTvDe:  Miscellaneous 

□Academic  Knowledge# 

□Aviation  Domain# 

□Collaboration# 

□Computer  Skills  (non-military)# 

□Equipment  Diagnostics  8i  Maintenance# 
□Equipment  Usage# 

□Laboratory  Task# 

□Managerial  # 

□Medical  Domain# 

□Science,  Math  or  Programming  Skills# 
□Situation  Assessment# 

□Sports# 

□Target  Identification  # 

□Tracking# 

□Vehicle  Control  # 

'  □  Web- based  # 

Figure  38.  Filter  by  Attributes 

Definitions  of  each  attributes  are  shown  in  Appendix  A.  You  can  also  view  attribute 
definitions  by  hovering  your  mouse  over  the  “i”  to  the  right  of  any  attribute. 

Once  all  desired  attributes  have  been  selected,  left  click  the  “Search”  button  at  the  bottom 
right  of  the  attribute  window.  A  filtered  list  of  studies  is  displayed  in  the  scrolling  list  at  the 
bottom  of  the  window. 
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Export  to  Excel 


You  can  export  the  document  list  contents  to  excel  by  clicking  the  “Export  to  Excel.” 
TARGET  will  create  a  file  named  grid.xls  in  the  download  folder  of  your  computer  (by  default 
the  folder  is  c:\ETsers\wsername\Downloads).  The  exported  file  is  on  Microsoft  Excel  97-2003 
format.  The  excel  file  contains  all  of  the  fields  on  the  index  document  list  display  (Figure  39). 
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B  C  D  E  F  G 
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Title 

Authors 

Year 

Document  Source 

Report  Type 

Date  Created 

Created  By 

2 

3 

4 

5 

6 

The  training  of  complex  task  performance 

Fabiani,  M.;  Buckley,  J.;  Gratton,  G.;  Coles,  M.; 
Donchin,  E.;  Logie,  R. 

1989 

Acta  Psychologica 

Experimental  Data 

9/8/2010  15:41 

cvoeller 

Contributions  of  a  part-task  trainer  to  the 
learning  and  relearning  of  a  time-shared 
flight  maneuver 

Adams,  J.  A.;  Hufford,  L  E. 

1962 

Human  Factors:  The  Journal  of  the  Human 
Factors  and  Ergonomics  Society 

Experimental  Data 

9/7/2010  7:41 

cvoeller 

Part-whole  practice  of  movement  sequences 

Park,  J.  H.;  Wilde,  H.;  Shea,  C.  H. 

2004 

Journal  of  Motor  Behavior 

Experimental  Data 

9/13/2010  23:56 

cvoeller 

The  relative  efficiency  of  several  training 
methods  as  a  function  of  transfer  task 
complexity 

Briggs,  G.  E.;  Naylor,  J.  C. 

1962 

Journal  of  Experimental  Psychology 

Experimental  Data 

9/7/2010  7:58 

cvoeller 

Transfer  of  skill  from  a  computer  game 
trainer  to  flight 

Gopher,  D.;  Weil,  M.;  Bareket,  T. 

1994 

Human  Factors:  The  Journal  of  the  Human 
Factors  and  Ergonomics  Society 

Experimental  Data 

7/9/2013  7:07 

cvoeller 

Figure  39.  Excel  Export  File 


Document  Details 

To  modify  or  view  further  details  of  any  individual  document,  first  select  the  row  that  the 
desired  document  is  on,  then  select  the  “Edit”  or  “Details”  hyperlink  at  the  right  side  of  each 
study  (Figure  40).  To  delete  a  document  click  the  “Delete”  hyperlink.  Note:  as  a  contributor 
you  will  only  be  able  to  edit  and  delete  documents  that  you  created. 


tP  Delete  Details  l°  Edit 

Figure  40.  Document  Hyperlink  Menu 

Selecting  “Edit”  or  “Details”  will  display  details  about  the  individual  document.  The 
“Details”  page  contains  the  following  text  fields:  title,  authors,  publication  year,  citation, 
abstract,  document  type,  data  source,  report  type,  and  document  creator  (Figure  41). 

Hyperlinks  are  available  at  the  bottom  of  the  window  for  different  types  of  document 
information:  Summary  Details,  Attribute  Details,  Contrasts,  and  Back  to  List  (Figure  41).  Edit  is 
also  available  if  you  created  the  record  or  are  an  Administrator  user  type. 
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Details 

f  Documentj 

Document  Title: 

Human  factors  simulation  in  construction  management  education 

Author(s): 

Jaeger,  M.  Adair.  D. 

Year: 

2010 

Citation: 

Jaeger,  M.  &  Adair,  D.  (2010).  Human  factors  simulation  in  construction  management  education.  European  Journal  of  Engineering  Education,  35(3),  299-309. 

Abstract: 

Successful  construction  management  depends  primarily  on  the  representatives  of  the  involved  construction  project  parties.  In  addition  to  effective  application  of  construction 
management  tools  and  concepts,  human  factors  impact  significantly  on  the  processes  of  any  construction  management  endeavour.  How  can  human  factors  in  construction  management 
be  taught  effectively?  Although  simulations  are  applied  in  construction  management  education,  they  have  not  incorporated  human  factors  sufficiently.  The  focus  on  human  factors  as  part 
of  the  simulation  of  construction  management  situations  increases  students'  learning  effectiveness  within  a  cross-cultural  teaching  setting.  This  paper  shows  the  development  of  discrete- 
event  human  factors  in  construction  management  simulation.Adescription  of  the  source  code  is  given.  Learning  effectiveness  in  a  cross-cultural  education  setting  was  analysed  by 
evaluating  data  obtained  through  student  questionnaire  surveys.  The  mean  score  obtained  by  the  students  using  the  simulator  was  32%  better  than  those  not  exposed  to  the  simulator.  The 
spread  of  results  was  noticeably  greater  for  the  students  not  exposed  to  the  simulator.  The  human  factors  simulation  provides  an  effective  means  to  teach  students  the  complexities  and 
dynamics  of  interpersonal  relationships  in  construction  management. 

Document  Type: 

Journal  Article 

Data  Source: 

European  Journal  of  Engineering  Education 

Report  Type: 

Experimental  Data 

Created  By: 

cvoeller 


Edit  |  Summary  Details  |  Attribute  Details  |  Contrasts  |  Back  to  List 


Figure  41.  Example  Details  Page  with  Hyperlink  menu  at  the  bottom 
Summary  Details 

The  “Summary  Details”  link  takes  you  to  the  Document  Summary  page  that  contains  the 
following  text  fields:  brief  research  study  description,  the  training  methods  reviewed  in  the 
research  study,  the  tasks/skills  reviewed  in  the  study,  the  theoretical  framework  and/or 
conceptual  framework  of  the  study,  overall  findings,  training  method  specific  findings,  findings 
relevant  to  outcome  criteria,  findings  relevant  to  task/skill  differences,  findings  relevant  to 
individual  differences,  other  findings,  author  conclusions,  and  user  comments  (Figure  42). 
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Document  Summary 

Document  Title:  Human  factors  simulation  in  construction  management  education 
Author(s):  Jaeger,  M.;  Adair,  D. 

Yean  2010 

Report  Type:  Experimental  Data 


Article  Description  j 

A  prototype  training  tool  was  developed  to  be  part  of  a  construction  management  module  at  the  Australian  College 
of  Kuwait  (ACK).  The  tool  is  a  computer-based  program  that  the  authors  call  a  "simulation"  and  its  intend  purpose  is 
to  teach  "human  factors".  In  this  case  "human  factors"  refers  to  what  is  more  often  referred  to  as  interpersonal  skills. 
The  "simulation"  is  a  set  of  scenarios  (i.e.  vignettes).  For  each  scenario  the  student  must  read  about  the  provided 
information  regarding  the  situation  and  select  what  he  thinks  is  the  best  course  of  action.  Once  a  course  of  action  is 
selected  the  simulation  provides  immediate  feedback  regarding  the  correctness  of  his  select  course  of  action  and  if 
the  action  was  incorrect  what  the  student  might  want  to  think  about  before  choosing  a  new  course  of  action.  The 
authors  pilot  tested  the  training  tool  to  determine  its  effectiveness. 


Training  Method  Definition: 

Simulation  (Computer-based):  "It  should  incorporate  specifics  of  the  local  as  wells  as  the  global  construction  sector 
. Real  situations...simulated  in  a  descriptive  way."  (p.  301) 

Interactive  with  feedback:  The  program  should  have, "...  various  potential  reactions  to  choose  from  and  immediate 
and  encouraging  feedback."  The  feedback  for  an  incorrect  choice  should  be, "...  combined  with  encouragement  to 
rethink  their  input."  (p.  301) 

Self-paced:  Having  the  application  be  self-paced  was  a  necessity  since  some  students  using  the  program  would  have 
English  as  a  second  language  and  would  need  time  to  look  up  words  and  definitions. 


Target  Task/Skill  Description  > 

Construction  Management: "...  a  management  discipline  that  depends  foremost  on  the  construction  manager's 
personality  and  the  personalities  of  the  representatives  of  the  involved  project  parties."  (p.  300) 

Human  Factors  (a.k.a  Interpersonal  Skills): "...  social  competence,  communication  abilities,  situational  awareness,  learn 
ability,  attention,  reliability,  ability  to  work  under  pressure,  decision-making  skills,  work  attitude,  critical  thinking, 
leadership  ability,  assertiveness,  vision,  balance,  etc.  are  influenced  by,  amongst  others,  heredity  transmission,  culture, 
education,  environment  and  world  view."  (p.  300). 

Conflict  Resolution  and  cultural  awareness  were  the  primary  IPS  skills  focused  on  in  the  article. 


Theoretical/Conceptual  Approach  : 

Figure  42.  Document  Summary  Page 


The  fields  are  described  in  Table  2  and  can  be  seen  by  hovering  the  mouse  over  the  “i”  next 
to  any  field. 
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Table  2.  Summary  Field  Descriptions 


Field 

Description 

Article  Description 

A  free  text  field  designated  for  a  high  level  summary  of  the  document,  e.g., 
purpose,  what  was  examined,  and  what  was  found 

Training  Method 
Definition 

A  free  text  field  designated  for  a  description  of  the  training  methods 
examined/described  in  the  document 

Target  Task/Skill 
Description 

A  free  text  field  designated  for  a  description  of  the  tasks  or  skills  addressed  in  the 
document. 

Theoretical/Conceptual 

Framework 

A  free  text  field  designated  for  a  description  of  the  conceptual  framework  or 
theory  that  frames  the  content  of  the  document,  if  relevant 

Overall  Findings 

A  free  text  field  designated  for  description  of  the  main  findings  of  the  document, 
if  relevant 

Findings  by  Training 
Method 

A  free  text  field  designated  for  description  of  specific  findings  relative  to  the 
training  method(s)  examined  in  the  document 

Findings  by  Outcome 
Criteria 

A  free  text  field  designated  for  description  of  specific  findings  relative  to  the 
outcomes  measured  in  the  document. 

Findings  by  task  or  skill 
type 

A  free  text  field  designated  for  description  of  specific  findings  relative  to  the  task 
or  skill  categories  examined  in  the  document 

Findings  by  Individual 
Factors 

A  free  text  field  designated  for  description  of  specific  findings  relative  to  the 
individual  differences  between  study  participants,  e.g.,  experience,  ability,  or  age 

Other  relevant  findings 
or  guidelines 

A  free  text  field  designated  for  description  of  additional  findings  relevant  to 
training  that  are  not  captured  above  under  findings  by  method,  task/skill,  or 
individual  differences 

Author  Conclusions 

A  summary  of  the  key  points  summarized  by  the  document  author(s) 

Reviewer  Comments 

A  free  text  field  designated  for  the  user/creator  to  add  relevant  comments  or 
information  regarding  the  document 

Attribute  Details 

The  “Attribute  Details”  link  takes  you  to  the  Document  Attribute  page  that  shows  you  all 
applicable  attributes  coded  for  this  document.  The  document  attributes  are  the  same  ones  that 
are  available  to  filter  documents  with  or  to  code  a  document  for  under  the  “Add  New  Document” 
feature  (Figure  43).  For  descriptions  of  each  attribute  see  Table  2  or  hover  the  mouse  over  the 
“i”  next  to  each  attribute. 
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Document  Attributes 


Document  Title;  Human  factors  simulation  in  construction  management  education 
Author(s):  Jaeger,  M.;  Adair,  D. 

Yean  2010 

Report  Type;  Experimental  Data 


Training  Method:  Practice 

Group  Practice  y 
Mental  Practice  y 
Overlearning  y 
Practice  Spacing  y 
'  Scenario- Based  Practicey 

Training  Method:  Managing  Difficulty 

Adaptive  Training  y 
Exploration  j) 

Increasing  Difficulty  i) 

Learner  Control  y 
Part  Task  Training  y 
Scaffolding  0 

Training  Wheels/Error  Prevention^ 
Variable- Priority  Training  y 
Worked  Examples  ;; 

Training  Method:  Other 

Behavior  Modeling  y 
' Feedback y 
Goal  Based  y 

Prior  knowledge  activationy 
Problem  Based  Learning  y 
Proceduralized  Instruction  <y 
Team  Training  D 


Task  Category 

Cognitive-Complex  Decision  Making  *y 
Cognitive- Declarative  <j) 

'  Cognitive- Problem  Solving  y 
Cog  ni tive-  Proced u  ral  y 
Cognitive-Quantitative  reasoning  y 
Cognitive- Spatial  Reasoning  y 
'  Interpersonal  <]) 

Perceptual  y 
Psychomotor  (]) 

Task  Type:  Military  Specific 

Adaptability  ;; 

Battlespace  Awareness  ;j 
Command  and  Control  <]) 

Critical  Thinking]) 

Digital  Skillsy 
Intelligence  Gathering  y 

Task  Type:  Interpersonal  Skills 

Active  Listening]) 

Assertive  Communication  y 
'Conflict  Resolution]) 

'Cultural  Awarenessy 
Leadership  Skillsy 
Negotiationy 

Nonverbal  Communication  y 
Relationship  Building  y 

Task  Type:  Miscellaneous 

Academic  Knowledge;; 

Aviation  Domainy 
Collaboration  y 

Computer  Skills  (non-military)]) 
Equipment  Diagnostics  &  Maintenance;; 
Equipment  Usage  ;; 

Laboratory  Task]) 

Managerial  y 
Medical  Domain]) 

Science,  Math  or  Programming  Skillsy 
Situation  Assessment  y 
Sports  y 

Target  Identification  < y 
Tracking  y 
Vehicle  Control  y 


Individual  Factors 

Ability  y 
OAgey 

Education  Level  y 
Experience y 
Motivation  y 

Outcome  Measure 

Learner  Reaction  y 
i'  Learning  y 
Retentiony 
Transfer;; 

Transfer-Joby 

Meta-analysis 

Training  Difficultyy 

Sample 

□Academic;; 

Military]) 

'Work;; 

Delivery  Method 

Classroom]) 

Comparing  Delivery  Methods  y 
'  Computer-based  y 
Field/Real  System  y 
Gaming]) 

'  Simulation-based  y 
Web- based  y 


Edit  Attributes  |  Back  to  Document 


Figure  43,  Document  Attributes  Page 
Link  to  Add  New  Document  function 

The  Add  New  Document  function  links  to  the  component  for  adding  new  user  documents 
(e.g.,  a  new  research  study).  To  add  a  new  document,  left  click  the  “Create_New”  hyperlink  in 
the  top  left  of  the  document  list  window  (Figure  44).  The  link  will  take  you  to  the  first  page  of 
the  component  for  adding  new  user  documents  to  the  database.  Once  selected,  you  will  switch 
pages  to  the  Add  New  Document  component  where  you  can  begin  adding  document  level 
attribute  information  such  as  the  title,  authors,  document  type,  etc. 


Create  New  y 

Figure  44.  Create  New  Hyperlink 
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Adding  New  Documents 


Users  can  enter  infonnation  from  new  documents  (e.g.,  a  new  research  study)  into  the 
research  database  and  compare  the  findings  with  the  infonnation  already  in  TARGET. 

Overview  of  Add  New  Document  Feature 

The  Add  New  Document  component  of  TARGET  provides  users  with  the  capability  to  add 
new  document/research  studies  to  the  database  and  keep  TARGET  up  to  date  with  the  latest 
literature.  New  studies  can  be  document  coded  only,  document  coded  with  a  summary, 
document  coded  with  study  data  for  effect  size  computations,  or  all  three.  The  Add  New 
Document  component  offers  four  functions:  (1)  Create  document,  (2)  Code  document  attributes, 
(3)  Add  document  summary,  and  (4)  Add  contrasts  for  effect  size  calculations. 

To  add  a  new  document  from  the  home  page,  select  the  blue  “Add  New  Document”  link. 

This  component  is  also  available  through  the  “Create  New”  tab  at  the  top  of  the  Documents 
page.  This  feature  is  available  to  all  users. 

Creating  a  New  Document 

The  first  window  of  the  “Add  New  Document”  component  presents  a  set  of  fields  that  are  the 
minimum  required  document  attributes  that  must  be  entered  in  order  to  create  a  new  document 
(Figure  45). 


Create 


Figure  45.  Create  Window 
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Creating  a  new  document  is  the  first  step  in  adding  a  new  document  to  the  database.  Here, 
basic  information  for  a  new  document  is  entered  and  saved.  The  minimum  information 
requirements  to  create  a  new  document  are  as  follows: 

•  Title  -  The  title  of  the  published  document. 

•  Author(s)  -  All  author  names,  in  the  order  listed  on  the  publication,  entered  in 
standard  APA  format  last  name,  first  initial. 

•  Year  -  The  year  the  document  was  published. 

•  Document  Type  -  Identifies  the  document  source  as  a  journal  article,  book, 
conference  paper,  etc. 

•  Data  Source  -  Identifies  the  name  of  the  publication,  such  as  Journal  of  Educational 
Psychology,  Journal  of  Applied  Psychology,  etc. 

•  Report  Type  -  Identifies  the  type  of  publication,  such  as  a  Meta-Analysis, 
Experimental  paper,  Literature  review,  etc. 

•  Citation  -  Standard  APA  format  citation  including  authors,  year,  title,  and  source 
information. 

•  Abstract  -  Cut  &  paste  from  publication  if  available.  Typically  includes  a  summary 
of  main  points  such  as:  objective,  background,  method,  results,  and  conclusion. 

Two  fields  are  automatically  generated  by  TARGET  when  you  create  a  new  document. 

•  Reviewed  By  -  Identifies  who  created  the  study.  This  field  is  automatically 
populated  with  the  creator’s  username. 

•  Date  Created  -  Identifies  the  date  that  the  document  was  created.  This  field  is 
automatically  populated  by  the  tool. 

The  user  clicks  the  “Create”  button  to  create  the  document  in  the  database.  Once  the 
document  has  been  created,  the  user  may  code  additional  document  attributes,  add  qualitative 
document  summary  information,  define  specific  contrasts  for  effect  size  calculations,  code  the 
study-level  attributes  specific  to  each  contrast,  select  the  type  of  data  to  enter  for  each  contrast, 
and  enter  statistics  for  effect  size  computation. 

Once  created,  TARGET  will  display  the  Document  Details  page  showing  the  infonnation 
you  just  entered.  Now,  you  may  choose  to  edit  the  details,  create  summary  info,  code  for 
attributes,  add  contrasts  for  effect  size  calculations  using  the  menu  at  the  bottom  of  the  page,  or 
go  back  to  the  document  list  (Figure  46). 


Edit  |  Create  Summary  |  Create  Attributes  |  Contrasts  |  Back  to  List 


Figure  46.  Document  Details  page  Hyperlink  menu  options 
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Adding  a  New  Author 

To  add  a  new  author,  click  on  the  “Add  an  Author”  button  -  TARGET  will  display  the  author 
drop-down  list.  Scroll  to  the  top  of  the  list  and  select  <New  Author>.  TARGET  will  display 
fields  to  enter  the  new  author’s  last  name  and  first  name  (Figure  47).  Enter  the  author’s  first  and 
last  names  into  the  appropriate  fields  and  click  the  “Create  New  Author”  button. 


Figure  47.  Add  New  Author 
Adding  a  New  Data  Source 

To  add  a  new  data  source,  scroll  to  the  top  of  the  “Data  Source”  drop-down  list  and  select 
<New  Data  Source>.  TARGET  will  display  a  field  to  enter  the  new  data  source  title.  Type  the 
name  of  the  new  data  source  and  click  the  “Create  New  Data  Source”  button. 


Figure  48.  Add  New  Data  Source 
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Add  Document  Summary 

Documents  may  also  be  characterized  by  qualitative  descriptions  of  the  research  study’s 
purpose,  design,  findings,  and  conclusions.  The  “Summary”  feature  under  Add  New  Document 
allows  the  user  to  summarize  elements  of  a  study  by  typing  in  summary  information  (Figure  49). 
The  feature  utilizes  free  text  fields  where  users  type  information  directly  into  the  field.  The 
available  free  text  fields  include: 

•  Article  description  -  A  free  text  field  designated  for  a  high  level  summary  of  the 
document  (e.g.,  purpose,  what  was  examined,  and  what  was  found). 

•  Training  methods  reviewed  -  A  free  text  field  designated  for  a  description  of  the 
training  methods  examined/described  in  the  document. 

•  Theoretical/conceptual  frameworks  -  A  free  text  field  designated  for  a  description  of 
the  conceptual  framework  or  theory  that  frames  the  content  of  the  document,  if 
relevant. 

•  Overall  findings  -  A  free  text  field  designated  for  description  of  the  main  findings  of 
the  document,  if  relevant. 

•  Findings  by  training  method  -  A  free  text  field  designated  for  description  of  specific 
findings  relative  to  the  training  method(s)  examined  in  the  document. 

•  Findings  by  outcome  criteria  -  A  free  text  field  designated  for  description  of  specific 
findings  relative  to  the  outcomes  measured  in  the  document. 

•  Findings  related  to  task/skill  type  differences  -  A  free  text  field  designated  for 
description  of  specific  findings  relative  to  the  task  or  skill  categories  examined  in  the 
document. 

•  Findings  related  to  individual  differences  -  A  free  text  field  designated  for  description 
of  specific  findings  relative  to  the  individual  differences  between  study  participants 
(e.g.,  experience,  ability,  or  age). 

•  Other  relevant  findings  or  guidelines  -  A  free  text  field  designated  for  description  of 
additional  findings  relevant  to  training  that  are  not  captured  above  under  findings  by 
method,  task/skill,  or  individual  differences. 

•  Author  conclusions  -  A  summary  of  the  key  points  summarized  by  the  document 
author(s). 

•  Reviewer  comments  -  A  free  text  field  designated  for  the  user/creator  to  add  relevant 
comments  or  information  regarding  the  document. 
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Create  Document  Summary 

Document  Title:  TEST 

Author(s):  Adair,  D. 

Year:  2013 

Report  Type:  Experimental  Data 

Article  Description  t 

Training  Method  Definition  ? 

Target  Task/Skill  Description  j 

Theoretical/Conceptual  Approach  • 

Study  Setting  > 

Study  Participants  i 

Training  Task  Description  i) 

Study  Design  j 

Figure  49.  Create  Document  Summary  Page 

Select  the  “Save”  button  to  save  the  qualitative  descriptions  in  the  database. 


Code  Additional  Document  Attributes 

A  large  number  of  additional  document  attributes  may  be  coded  for  the  new  document  once 
it  has  been  created.  To  code  additional  attributes,  click  on  the  “Create  Attributes”  link  at  the 
bottom  of  the  Document  Details  window  -  define  and  describe  all  available  attributes  (Figure 
50).  Select  desired  attributes  with  the  radio  buttons  and  check  boxes.  Definitions  for  all 
attributes  can  be  found  in  Appendix  A  -  Attribute  Definitions  or  by  hovering  your  mouse 
over  the  “i”  to  the  right  of  each  attribute. 
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Create 

Document  Attributes 


Document  Title:  TEST 

Author(s):  Adair,  D. 

Yean  2013 

Report  Type:  Experimental  Data 

Training  Method:  Practice 

Task  Category 

Individual  Factors 

□Group  Practice  i) 

□Cognitive-Complex  Decision  Making  ]p 

□Abilityp 

□  Mental  Practice]) 

□Cognitive-Declarative  i> 

□Agep 

□  Overlearning  p 

□Cognitive- Problem  Solving  ]) 

□  Education  Level  p 

□  Practice  Spacing  p 

□Cognitive- Procedural  ]) 

□  Experience :; 

□Scenario- Based  Practice]) 

□Cognitive-Quantitative  reasoning  0 
_ □Cognitive-Spatial  Reasoning  p 

□  Motivation  p 

Training  Method:  Managing  Difficulty 

Outcome  Measure 

□Adaptive  Training  p 

□  Perceptual  :) 

□  Learner  Reaction  p 

□  Exploration  p 

□  Psychomotor  ip 

Li  Learning  p 

□  Increasing  Difficulty!) 

□  Retentionp 

□  Learner  Control  p 

Task  Type:  Military  Specific 

□Transfer:) 

□  Part  Task  Training]) 

□Adaptability  p 

□Transfer-Jobp 

□Scaffolding  p 

□  Battlespace  Awareness p 

Meta-analysis 

□Training  Wheels/Error  Prevention  ]) 

□Command  and  Control  'p 

□Variable-Priority  Training  p 

□Critical  Thinking]) 

□Training  Difficulty.:) 

□Worked  Examples:) 

□  Digital  Skills]) 

Sample 

Training  Method:  Other 

U  Academic.:) 

□  Behavior  Modeling  p 

Task  Type:  Interpersonal  Skills 

□  Military]) 

□  Feedback  i> 

□Active  Listening  ]) 

□Work]) 

□Goal  Based  p 

I  'Assertive  Communication 

Delivery  Method 

□Classroom]) 

□  Prior  knowledge  activation  p 

□  Problem  Based  Learning  p 

□Conflict  Resolution  ]) 

□Cultural  Awareness:) 

□  Proceduralized  Instruction  <p 

□  Leadership  Skills  u 

□Comparing  Delivery  Methodsp 

□Team  Training  .:) 

□Negotiation]) 

U Computer- based  ■:) 

□Field/Real  System P 

□  Relationship  Building]) 

□Gaming  p 
□Simulation-based  p 

Task  Type:  Miscellaneous 

□Academic  Knowledge  u 
□Aviation  Domain]) 

□Collaboration  p 

□Computer  Skills  (non-military)  u 

□  Equipment  Diagnostics  &  Maintenance  :) 

□  Equipment  Usage:) 

□  Laboratory  Task]) 

□  Managerial  <]) 

□  Medical  Domain  !) 

□Science,  Math  or  Programming  Skills.:) 
□Situation  Assessment]) 

□Sports]) 

□Target  Identification  >]) 

□Tracking  p 
□Vehicle  Control  p 

□Web-based  u 

Save 

Back  to  Document  Details 

Figure  50.  Create  Attributes  Page 

Select  the  “Save”  button  in  the  bottom  left  comer  to  save  the  selected  attribute  codes  to  the 
database  for  the  new  document. 
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Define  Specific  Contrasts  for  Effect  Size  Calculations 

For  the  new  document,  users  may  also  define  one  or  more  specific  contrasts  to  be  used  in 
effect  size  computations  within  TARGET.  There  are  four  steps  to  adding  contrasts  with  effect 
size  data  to  the  database.  The  first  step  is  defining  the  contrasts  to  be  entered.  The  set  of 
contrasts  are  entered  under  the  “Contrasts”  link  on  the  Document  Details  Page.  This  will  take 
you  to  the  Contrasts  Index  page  (Figure  51)  where  you  can  click  the  “Create  New”  link  to  add 
one  or  more  new  contrasts. 


Index 

Create  New 

There  are  no  Contrasts  for  this  Document. 
Back  to  Document 


Figure  51.  Contrast  Index  Page 


Figure  52.  Create  New  Contrast:  Basic  Information 


On  the  Create  New  Contrast:  Basic  Information  page  (Figure  52),  the  minimum  information 
that  must  be  entered  to  define  a  contrast  includes: 

•  Short  Title  -  A  free  text  field  designated  for  a  brief  description  of  the  contrast  (e.g., 
Jones  et  al.  (2012),  high  experience,  pure  part  training  versus  low  experience,  pure 
part  training.) 

•  Description  (Optional)  -  A  free  test  field  that  is  optionally  available  for  entering  a 
longer  description  of  the  study  contrast  procedures  and/or  data. 
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•  Method  Type  -  A  drop-down  menu  where  the  user  specifies  the  training  method 
central  to  the  treatment  group  in  the  contrast.  In  TARGET,  a  contrast  is  considered  a 
comparison  of  treatment  group  versus  control  group,  primarily  defined  for  the 
purpose  of  creating  an  effect  size  estimate  for  the  magnitude  of  the  difference  in 
transfer  performance  between  the  two  groups.  Method  options  include: 

o  Part  task  training :  Training  method  that  decomposes  complex  tasks  into  a 
series  of  smaller  tasks,  each  of  which  is  demonstrated  and  practiced  separately 
before  being  practiced  as  a  whole  task. 

o  Increasing  difficulty :  Parameters  of  the  whole  task  are  initially  set  to  lower 
levels,  to  reduce  the  intrinsic  load  during  early  training  trials,  and  then 
increased  as  training  progresses,  until  the  difficulty  reaches  the  level  of  the 
whole  task.  The  difficulty  levels  can  increase  in  either  a  fixed  or  adaptive 
schedule. 

o  Learner  control :  An  instructional  method  that  provides  trainees  with  decision¬ 
making  control  over  specific  dimensions  or  activities  within  a  structured 
learning  environment. 

o  Exploratory  learning'.  A  trainee  explores  a  task  environment  on  his/her  own. 
The  level  or  type  of  guidance  given  to  the  trainee  vary  (i.e.,  only  a  user 
manual  vs.  conceptual  input  when  trainer  queried). 

o  Error  Prevention'.  An  error  prevention  strategy  geared  towards  reducing  the 
complexity  of  the  transfer  task  during  initial  learning  for  the  purpose  of 
reducing  training  task  errors,  and  to  help  trainees  acquire  the  appropriate 
schema  to  assimilate  the  transfer  task  throughout  learning.  Utilizes  hard 
lockouts  and  worked  examples. 

o  Scaffolding'.  Assistive  supports  are  provided  to  the  trainee  to  ease  the  demands 
of  independent  task  performance.  The  scaffold  supports  are  incrementally 
faded  out  over  time  until  the  trainee  is  executing  the  whole  task  independently 
with  minimal  demand. 

•  Transfer  Distance  -  A  drop-down  menu  where  the  user  specifies  the  transfer 
distance  measure  of  the  contrast.  Available  transfer  distance  options  are:  Identical 
tests,  near  transfer  test,  and  far  transfer  tests. 

o  Identical  tests  examine  the  application  of  what  was  learned  in  training  using  a 
task  or  problem  identical  to  that  used  during  training. 

o  Near  transfer  tests  examine  the  application  of  what  was  learned  in  training 
using  a  task  or  problem  somewhat  different,  but  similar,  to  that  used  during 
training. 

o  Far  transfer  tests  examine  the  application  of  what  was  learned  in  training 
using  a  testing  situation  that  is  different  or  new  from  the  task  or  problem  that 
was  used  in  training 
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Once  the  basic  contrast  infonnation  has  been  entered  click  the  “Next”  button  to  save  your 
entries.  If  you  decide  not  to  enter  the  new  contrast  click  the  “Cancel  Create  New  Contrast” 
hyperlink  at  the  bottom  left  of  the  page.  Once  contrasts  have  been  defined,  you  may  then  enter 
specific  moderator  parameters. 

Code  the  Study-Level  Moderators  Specific  to  Each  Contrast 

In  addition  to  the  basic  infonnation  that  must  be  added  for  a  new  contrast  (i.e.,  description 
field,  training  method,  and  transfer  distance),  you  may  code  the  contrast  for  more  specific 
infonnation.  This  is  the  second  step  for  adding  a  new  contrast  to  the  database.  Here  you  provide 
moderator  attribute  coding  to  represent  the  experimental  manipulations  that  define  the  specific 
control  versus  treatment  contrast  defined  in  the  prior  step.  If  you  are  adding  a  new  contrast,  once 
you  select  “Next”  from  the  basic  information  page,  TARGET  will  automatically  add  the  new 
contrast  and  take  you  to  the  Create  New  Contrast:  Moderators  page  (Figure  53).  Moderators  are 
selected  with  check  boxes  or  drop-down  lists. 

Create  New  Contrast:  Moderators 

■  New  Contrast:  testlj 

Method  Type:  Learner  Control 

Instructions:  Enter  details  about  the  contrast  so  that  it  can  be  classified  by  moderator  on  the  Forest  Plots 

Task  Category  (Choose  up  to  3)  i _ 

□  Perceptual  □  Psychomotor  □  Interpersonal 

□  Cognitive- Declarative  □  Cognitive- Procedural  □  Cognitive- Problem  Solving 

□  Cognitive-Spatial  Reasoning  □Cognitive-Quantitative  Reasoning  Q Cognitive-Complex  Decision  Making 


Difficulty  i) 


|  Select  a  Difficulty  |  ▼  | 

Ability  p 

|  Select  an  Ability  |^| 

Experience  i> 

|  Select  an  Experience  M 

Transfer  Measure  § 

|  Select  a  Transfer  Measure 

33 

Transfer  Test  p 

|  Select  a  Transfer  Test  !▼! 

Control  Vs  Treatment  Contrast  i 

|  Select  a  Control  Vs  Treatment  Contrast  [T| 

Control  And  Treatment  Condition  p 

I  Select  a  Control  And  Treatment  Condition  |T] 


Next 


Cancel  Create  New  Contrast 


Figure  53.  Create  New  Contrast:  Moderators  Page 
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The  user  may  code  the  following  moderator  attributes  prior  to  moving  to  the  third  step  and 
entering  data: 

•  Task  Category  -  The  skills  that  characterize  the  trained  task.  Users  may  check  up  to 
3  different  categories  from  among  the  following  9  options: 

o  Perceptual :  Skills  involved  in  detecting  and  interpreting  sensory  infonnation 
in  order  to  gain  awareness  and  support  performance.  The  ability  to  detect  and 
use  sensory  cues.  Include  some  perceptual  reasoning  tasks. 

o  Psychomotor.  Physical  skills  -  movement,  coordination,  manipulation, 

dexterity,  strength,  and  speed;  includes  both  fine  motor  skills  and  gross  motor 
skills. 

o  Interpersonal'.  Social  interaction  skills:  communicate,  exchange  infonnation, 
persuade,  build  and  maintain  relationships,  manage  conflict,  and  interact 
effectively. 

o  Cognitive-Declarative'.  This  category  includes  all  verbal  knowledge  categories 
including  facts,  principles,  knowledge  organization,  and  verbal  or  graphical 
mental  models,  concept  maps,  etc. 

o  Cognitive-Procedural'.  This  category  is  meant  to  include  perfonnance  of  basic 
procedural  skills,  where  the  procedure  may  include  cognitive  steps  and  basic 
perceptual  and  motor  steps  such  as  those  involved  in  navigating  a  computer 
interface.  Constrained  sequences  of  physical  and  cognitive  activities 
perfonned  in  predictable  situations. 

o  Cognitive-Problem  Solving:  This  category  involves  application  of  principles, 
rules  and  concepts  to  process  infonnation  and  solve  problems.  Includes 
general  logical  reasoning  skills:  inductive,  deductive,  diagnostic,  etc. 

o  Cognitive-Spatial  Reasoning:  Visual-spatial  skills  -  skill  in  representing, 
transforming,  generating,  and  recalling  symbolic,  nonlinguistic  infonnation. 
Includes:  mental  rotation;  spatial  perception  -  ability  to  determine  spatial 
relationships  with  respect  to  own  body  orientation;  spatial  visualization  - 
multi-step  manipulations  of  spatially  presented  information,  requiring  analysis 
of  the  relationship  between  different  spatial  representations. 

o  Cognitive-Quantitative  Reasoning:  Application  of  mathematical  concepts  and 
skills  to  solve  problems. 

o  Cognitive-Complex  Decision  Making:  This  category  involves  situation 
assessment  and  decision  making  in  complex,  dynamic  and  time-sensitive 
environments  with  changing  situations,  attentional  demands,  application  of 
strategies,  multiple  goals,  and  may  involve  coordination  of  perceptual  and 
motor  skills. 

•  Difficulty  -  A  general  category  of  subjective  task  difficulty  and  objective  task 
complexity;  typically  involves  an  experimental  manipulation  of  task  difficulty. 

o  Low:  An  experimental  condition  where  the  authors  manipulated  task  difficulty 
to  be  low;  or  the  training  task  clearly  has  no  interacting  variables,  making  it 
not  complex. 
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o  High :  An  experimental  condition  where  the  authors  manipulated  task 
difficulty  to  be  high;  or  the  training  task  clearly  has  interacting  variables, 
making  it  more  complex. 

•  Ability  -  An  individual  difference  variable  assessing  general  or  task-related  ability, 
skill,  or  aptitude. 

o  Low.  Participants  were  assessed  or  tested  as  having  low,  often  less  than 
median,  general  or  task-related  ability,  skill,  or  aptitude, 
o  High :  Participants  were  assessed  or  tested  as  having  high,  often  greater  than 
median,  general  or  task-related  ability,  skill,  or  aptitude. 

•  Experience  -  An  individual  difference  variable  assessing  prior  knowledge, 
familiarity,  or  practice  with  the  training  content. 

o  Low.  Participants  were  assessed  or  screened  through  sampling  as  having  little 
to  no  prior  knowledge,  familiarity,  or  practice  with  the  content, 
o  High:  Participants  were  assessed,  screened  through  sampling,  or  trained  up  to 
having  moderate  to  significant  prior  knowledge,  familiarity,  or  practice  with 
the  content. 

•  Transfer  Measure  -  The  type  of  outcome  measure  with  regard  to  measurement  of 
speed  or  accuracy  -  like  perfonnance. 

o  Accuracy/Quality:  Execution  of  appropriate  action  or  response  (e.g.,  steps, 
Multiple  Choice  /  True  or  False  /open  ended  answers,  problems,  recall, 
actions)  measured  as  the  percent  correct,  #  correct,  accumulated  points, 
task/test  scoring  correct  action/timing/performance  but  not  rating,  or  general 
appropriateness  of  action/response.  Also  include  quality  rating  by  subject 
matter  experts  (SMEs)  or  inverse  of  error  perfonnance. 
o  Performance/Response  time :  Time  to  respond  or  take  appropriate  action 
measured  as  time  to  complete  the  entire  transfer  task  (i.e.,  time  to  complete), 
time  to  complete  a  subtask  (i.e.,  time  to  respond),  time  to  reach  a  criterion 
level  of  perfonnance  (i.e.,  time  to  criterion),  or  trials  to  reach  a  criterion  level 
of  performance  (i.e.,  trials  to  criterion). 

•  Transfer  Test  -  Characterizes  the  nature  of  the  transfer  test  based  on  the  focus  of 
the  assessment. 

o  Knowledge :  Knowledge  tests  capture  the  ability  of  a  trainee  to  recall 
infonnation  or  problem  solve  from  a  trained  skill, 
o  Performance :  Performance  tests  capture  the  ability  of  a  trainee  to  physically 
performance  a  trained  skill. 

Several  method  specific  attributes  are  also  available.  These  options  are  displayed  at  the 
bottom  of  the  “Contrast  Parameters”  tab  (Figure  53).  Each  method  specific  set  of  attributes  is 
defined  below. 
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Part  Task  Training 

•  Part  task  manipulation:  The  training  treatment  manipulation  in  which  the  whole  task 
to  be  learned  is  somehow  decomposed  into  subtasks  and  trained  on  those  subtasks. 

o  Pure  part :  All  parts  are  trained  individually  before  any  integration  into  the 
whole  or  partially  whole  task. 

o  Cumulative  part :  Cumulative-part  training  [Repetitive-part  training]  is  where 
an  individual  part  is  practiced,  then  that  first  part  and  another  part  practiced, 
then  those  two  with  a  third  is  practiced,  and  so  forth  (i.e.,  1,  then  1&2,  then 
1&2&3,  then  1&2&3&4,  ...  then  whole  task). 

o  Progressive  part :  Progressive-part  training  is  where  each  part  is  first  practiced 
by  itself,  then  combined  with  a  progressively  more  integrated  whole  task  (i.e., 
1,  then  2,  then  1&2,  then  3,  then  1&2&3,  then  4,  then  1&2&3&4,  ...  then 
whole  task). 

o  Variable  priority  training :  VPT  is  where  training  occurs  with  the  whole  task 
but  under  conditions  of  particular  emphasis  of  participant  focus  and 
perfonnance  is  on  specific  sub  tasks  of  the  whole  task.  Take  as  an  example  the 
skill  of  batting  a  ball  in  baseball.  One  approach  would  be  to  practice  the  skill 
of  hitting  a  baseball  by  spending  an  hour  each  day  practicing  the  entire  swing 
and  trying  to  hit  the  ball  in  the  field.  This  would  be  consistent,  or  fixed 
priority  (FP)  training.  Another  approach  would  be  to  divide  the  hour  into  15 
minute  segments  with  an  emphasis  on  a  different  aspect  of  the  swing  (e.g. 
footwork,  follow  through,  initial  bat  placement,  and  stance),  all  while  still 
completing  the  entire  stroke. 

o  Other.  Any  less  typical  part  task  training  manipulations  with  partial  task 
practice  versus  a  control  condition  of  whole  task  practice. 

•  Concurrence:  The  degree  that  subtask  “parts”  are  timeshared  in  the  whole  task. 

o  Segmentation-.  Trained  subtasks  are  clearly  sequential  parts  (e.g.,  like  learning 
the  approach  for  landing  an  aircraft  and  then  actually  landing  the  aircraft,  or 
learning  the  easier  passage  then  the  harder  passage  in  a  musical  piece).  An 
example  of  this  is  sometimes  called  backward  chaining,  forward  chaining,  or 
chaining. 

o  Fractionation  -  integrated :  Fractionation  refers  to  the  two  (or  more)  trained 
sub-parts  that  are  performed  concurrently  in  the  whole  task  (e.g.,  like  training 
how  to  fly  a  plane  by  training  vertical  control  and  lateral  control  as  two  parts. 
A  good  pilot  will  time-share  these  two  when  doing  a  climbing  turn).  Another 
example:  learning  left  hand  and  right  hand  on  the  piano,  or  strumming  and 
chording  on  the  guitar.  Integrated  means  that  within  the  whole  task  (now 
consisting  of  two  time-shared  parts),  the  parts  interact  with  each  other;  that  is, 
they  are  integrated  tasks.  An  example  of  interaction  is  shifting  gears  versus 
using  the  clutch.  Although  each  may  be  practiced  separately,  when  actually 
changing  gears,  the  timing  of  the  two  components  needs  to  be  exquisitely 
integrated. 
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o  Fractionation  -  separated :  For  explanation  of  Fractionation,  see  previous 
paragraph.  Separated  means  that  within  the  whole  task  (now  consisting  of  two 
time-shared  parts),  the  parts  do  not  interact  with  each  other;  that  is,  they  are 
separate  tasks.  An  example  of  separated  is  talking  on  the  cell  phone  while 
driving.  Although  these  two  may  interfere  with  each  other  via  human  attention 
limitations,  within  the  system,  what  is  said  on  the  phone  has  no  influence  in 
how  the  car  responds. 

Increasing  Difficulty 

•  Control  -  Treatment  contrast:  The  comparative  training  treatment  manipulation  of 
increasing  difficulty  conditions. 

o  Constant  Difficulty  vs.  Adaptive  Steps:  Control  is  learner  independent  and  at  a 
constant  level  of  difficulty  versus  a  Treatment  of  any  of  learner  dependent 
adaptive  conditions.  Under  constant  difficulty  all  learners  receive  the  same 
schedule  of  fixed  step  increases.  Under  adaptive  steps,  difficulty  increases  for 
each  learner  are  done  on  the  basis  of  his  or  her  trial  by  trial  perfonnance. 
o  Constant  Difficulty  vs.  Fixed  Steps:  Control  is  learner  independent  and  at  a 
constant  level  of  difficulty  versus  Treatment  is  learner  independent,  increasing 
steps  in  difficulty  are  on  a  fixed  schedule  of  increasing  difficulty, 
o  Fixed  Steps  vs.  Adaptive  Steps:  Control  is  learner  independent  using  fixed 
steps  versus  a  Treatment  of  any  of  learner  dependent  adaptive  conditions. 

•  Contrast  detail:  Further  description  of  the  specific  instance  of  adaptive  training  used 
in  the  contrast. 

o  Constant  vs.  Adaptive:  Performance:  Control  is  learner  independent  and  at  a 
constant  level  of  difficulty  versus  Treatment  is  learner  dependent  -  difficulty 
steps  adapted  based  on  participant's  individual  or  groups’  perfonnance  or 
progress  on  the  training  task. 

o  Constant  vs.  Adaptive:  Initial  Differences:  Control  is  learner  independent  and 
at  a  constant  level  of  difficulty  versus  Treatment  is  learner  dependent  - 
pretesting  to  determine  initial  ability  and  then  “adaptively”  assigned  to  a 
starting  difficulty  level  based  on  pretest  perfonnance. 
o  Constant  vs.  Fixed  Steps:  Control  is  learner  independent  and  at  a  constant 
level  of  difficulty  versus  Treatment  is  learner  independent,  increasing  steps  in 
difficulty  are  on  a  fixed  schedule  of  increasing  difficulty, 
o  Fixed  Steps  vs.  Adaptive:  Performance:  Control  is  learner  independent, 
increasing  steps  in  difficulty  are  on  a  fixed  schedule  of  increasing  difficulty 
versus  Treatment  is  learner  dependent  -  difficulty  steps  adapted  based  on 
participant's  individual  or  groups'  performance  or  progress  on  the  training 
task. 

o  Fixed  Steps  vs.  Adaptive:  Initial  Differences:  Control  is  learner  independent, 
increasing  steps  in  difficulty  are  on  a  fixed  schedule  of  increasing  difficulty 
versus  Treatment  is  learner  dependent  -  pretesting  to  detennine  initial  ability 
and  then  “adaptively”  assigned  to  a  starting  difficulty  level  based  on  pretest 
performance. 
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o  Other.  General  category  for  all  other  adaptive  treatments.  Three  example 
adaptive  training  treatments  included  under  the  “other”  option  include: 

■  Self  adaptive:  Treatment  is  learner  dependent  -  difficulty  steps  adapted 
by  participant  at  participant's  discretion. 

■  Performance  maintenance:  Treatment  is  learner  dependent  - 
performance  or  error  rates  are  monitored  and  difficulty  is  adjusted  up 
or  down  to  maintain  a  desired  level  of  performance  that  is  typically 
staggered  in  increasingly  difficulty  blocks  of  trials. 

■  Adaptive  down:  The  initial  treatment  is  learning  independent,  but 
subsequent  trials  are  learner  dependent.  The  trainee  begins  with  a 
high-difficulty  trial  and  the  presentation  of  subsequent  training  trials 
depends  on  their  performance.  If  they  perform  poorly,  they  move  to  a 
less  difficult  trial.  If  they  perform  well,  they  move  to  a  more  difficult 
trial  (or  stay  at  the  same  level  of  difficulty  if  they  are  already  at  the 
high  difficulty  level). 

•  Adaptive  training  adjustments:  The  timing  constraint  by  which  training  difficulty  is 
adaptively  adjusted  more  or  less. 

o  Continuous  adjustments:  Difficulty  is  continuously  modified  by  real  time 
performance  monitoring  and  adjustments. 

o  Incremental  adjustments  by  trial:  Difficulty  is  adjusted  incrementally  (i.e., 
from  training  trial  to  trial)  based  on  previous  trial's  perfonnance. 

o  Incremental  adjustments  by  block:  Difficulty  is  adjusted  incrementally  (i.e., 
from  one  block  of  trials  to  the  next)  based  on  previous  block's  perfonnance. 

Learner  Control 

•  Control  -  Treatment  contrast:  The  comparative  training  treatment  manipulation  of  no 
(or  less)  learner  control  versus  learner  control. 

o  Programmatic  vs.  Full  learner  control:  Control  is  programmatic  (i.e.,  zero 
learner  control)  versus  Treatment  is  100%  learner  control  over  all  training 
dimensions. 

o  Programmatic  vs.  Partial  learner  control:  Control  is  programmatic  versus 
Treatment  only  provides  the  learner  with  control  over  one  or  a  subset  of 
training  dimensions. 

o  Partial  learner  control  vs.  Full  learner  control:  Control  only  provides  the 
learner  with  control  over  one  or  a  subset  of  training  dimensions  versus 
Treatment  is  100%  learner  control  over  all  training  dimensions. 

o  Partial  min  vs.  Partial  max:  Control  only  provides  the  learner  with  control 
over  one  training  dimension  versus  Treatment  provides  the  learner  with 
control  over  a  larger  range  of  training  dimensions. 

•  Partial  learner  control  dimensions:  The  specific  dimensions  of  learner  control  that 
trainees  had  control  over. 

o  Pace:  Learner  control  over  speed/pace  through  a  fixed  set  of  content  or 

material.  All  content  has  to  be  viewed,  but  time  to  progress  through  content  is 
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under  learner  control  through  either  unlimited  progression  time  or  choice  to 
repeat  content  modules  before  progressing, 
o  Branching :  Learner  control  over  path/sequence  through  a  fixed  set  of  content 
or  material.  All  content  has  to  be  viewed,  but  choice  over  order  is  under 
learner  control. 

o  Truncation  of  lessons :  Learner  control  over  stopping  point  in  training 

material,  regardless  of  exhaustiveness.  Specific  to  the  number  of  content  or 
lesson  “views,”  so  leaner  control  is  over  choice  to  view  a  subset  of  available 
content  lessons  (i.e.,  skipping  lessons  1  and  2  out  of  10  lessons), 
o  Truncation  of  feedback/practice:  Learner  control  over  practicing  content, 
taking  quizzes,  receiving  feedback  on  perfonnance,  or  viewing  interactive 
materials  in  an  additional  “view.”  Specific  to  the  learner’s  choice  over  an 
additional  “view”  that  is  not  lesson-oriented  but  an  additional  feedback  or 
learning  mechanism. 

o  Multiple'.  Any  combination  of  the  above  dimensions  of  learner  control, 
o  Other.  General  category  for  all  other  dimensions  of  learner  control  (e.g., 
learner  control  over  training  difficulty). 

Exploratory  Learning 

•  Level  of  Guidance  vs.  Exploration:  The  degree  of  learning  guidance  provided  to 
learners  versus  the  degree  of  self  exploration  of  the  training  content.  The  treatment 
condition  is  the  condition  that  maximizes  support  for  exploration  and  minimizes 
learning  content  guidance. 

o  Minimal  Guidance  vs.  Maximum  Exploration :  Control  is  minimal  instruction 
(orientation,  learning  goals,  objectives,  problem  description),  pretest  problem 
scenarios;  trial  specific  learning  objectives  or  tasks  to  perform  during 
exploration/discovery;  very  minimal  solicited  only  guidance  during 
exploration/practice,  more  conceptual  or  meta-cognitive  than  content  specific; 
unsolicited  guidance  only  when  needed  to  recover  from  error;  Knowledge  of 
results  but  no  corrective  feedback  versus  Treatment  is  unstructured  and 
unguided  exploration;  example  only  for  rule  discovery, 
o  Guided  vs.  Maximum  Exploration'.  Control  is  instruction  that  provides 

demonstration,  worked  example,  conceptual  framework,  solution  strategy  but 
not  in  a  detailed  step  by  step  approach;  predefined  well- structured  problems; 
access  to  worked  examples;  unsolicited  prompts  (instructor  or  computer 
based)  that  provide  content  advice  or  direction  or  prevent  errors;  limited 
corrective  guidance  during  delayed  (after  a  trial)  feedback  versus  Treatment  is 
unstructured  and  unguided  exploration;  example  only  for  rule  discovery, 
o  Guided  vs.  Minimal  Guidance :  See  above  for  Guided  Control  definition. 
Treatment  is  minimal  instruction  (orientation,  learning  goals,  objectives, 
problem  description),  pretest  problem  scenarios;  trial  specific  learning 
objectives  or  tasks  to  perfonn  during  exploration/discovery;  very  minimal 
solicited  only  guidance  during  exploration/practice,  more  conceptual  or  meta- 
cognitive  than  content  specific;  unsolicited  guidance  only  when  needed  to 
recover  from  error;  Knowledge  of  results  but  no  corrective  feedback. 
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o  Directed  vs.  Maximum  Exploration :  Control  is  detailed  step  by  step 
instructions,  demonstration,  worked  example  or  solution  path;  explicit 
descriptions  of  procedure  or  rule,  prior  to  or  during  practice  (or  after 
exploration  but  prior  to  transfer  testing);  immediate  corrective  feedback 
versus  Treatment  is  unstructured  and  unguided  exploration;  example  only  for 
rule  discovery. 

o  Directed  vs.  Minimal  Guidance :  See  above  for  Directed  Control  description. 
Treatment  is  minimal  instruction  (orientation,  learning  goals,  objectives, 
problem  description),  pretest  problem  scenarios;  trial  specific  learning 
objectives  or  tasks  to  perform  during  exploration/discovery;  very  minimal 
solicited  only  guidance  during  exploration/practice,  more  conceptual  or  meta- 
cognitive  than  content  specific;  unsolicited  guidance  only  when  needed  to 
recover  from  error;  Knowledge  of  results  but  no  corrective  feedback, 
o  Directed  vs.  Guided  Exploration :  See  above  for  Directed  Control  description. 
Treatment  is  instruction  that  provides  demonstration,  worked  example, 
conceptual  framework,  solution  strategy  but  not  in  a  detailed  step  by  step 
approach;  predefined  well-structured  problems;  access  to  worked  examples; 
unsolicited  prompts  (instructor  or  computer  based)  that  provide  content  advice 
or  direction  or  prevent  errors;  limited  corrective  guidance  during  delayed 
(after  a  trial)  feedback. 

•  Error  encouragement:  Whether  learners  are  encouraged  to  make,  and  learn  from, 
errors  or  mistakes  made  during  training. 

o  More  exploration  group  only.  The  treatment  group  with  greater  autonomy  to 
explore  is  the  only  group  encouraged  to  make  errors  during  training, 
o  No  or  less  exploration  group  only.  The  control  group  with  greater  training 
guidance  is  the  only  group  encouraged  to  make  errors  during  training, 
o  Neither.  Neither  treatment  nor  control  groups  are  encouraged  to  make  errors 
during  training. 

o  Both :  Both  treatment  and  control  groups  are  encouraged  to  make  errors  during 
training. 

Error  Prevention 

•  Type  of  training  wheels:  The  type  of  error  preventive  mechanism  used  in  the  training 
treatment. 

o  Lockouts:  Irrelevant  features  locked  out,  or  made  unavailable,  to  prevent 
common  errors. 

o  Worked  examples :  Worked  out  examples  of  the  training  problems  are 
presented  with  no  additional  guidance  or  infonnation. 
o  Worked  examples  with  prompts:  When  the  worked  out  example  is  presented,  it 
includes  prompts,  cues,  or  elaborations  relevant  to  the  correct  process/problem 
solution. 

•  Treatment  timing: 
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o  Instruction :  The  lockout  or  worked  example  is  only  imposed/available  during 
instruction,  prior  to  practice.  Instruction  starts  with  unsolved  problem  and 
then  fully  worked  out  example  presented  with  all  steps  shown  at  once,  not 
available  when  participant  practices  solving  the  problem, 
o  Practice :  The  lockout  or  worked  example  is  imposed/available  for  the  student 
to  reference  whenever  they  want  to  during  practice,  either  as  an  optional  view 
or  simply  the  first  few  out  of  a  several  problems  solved, 
o  Instruction  &  Practice :  The  lockout  or  worked  example  is  imposed/available 
both  during  instruction  and  practice. 

Scaffolding 

•  Scaffolding  Prompt  Type:  The  type  of  error  preventive  prompt  used  in  the  training 
treatment. 

o  Critical  thinking  about  the  concept :  Prompts  emphasizing  key  concepts  and 
relationships  critical  to  learning  the  training  content, 
o  Self-regulation  of  learning'.  Prompts  emphasizing  that  participants  monitor 
their  emerging  understanding  and  plan/engage  their  learning  experience, 
o  Strategy  based  prompts :  Prompts  emphasizing  underlying  principles,  rules, 
and  processes  required  to  solve  the  problem, 
o  Other.  A  general  category  for  all  other  types  of  prompts. 

•  Scaffolding  Administration:  Manipulations  involving  the  removal  schedules  of  the 
error  preventative  mechanism. 

o  Fixed :  Scaffolding  support  removed  or  added  on  a  fixed  schedule  independent 
of  the  participant's  performance  in  the  training, 
o  Adaptive:  Scaffolding  support  removed  or  added  adaptively  based  on 
participant  performance  in  the  training. 

•  Scaffolding  Delivery:  The  mode  of  delivery  of  the  scaffolding  support. 

o  Human  Tutor:  The  error  preventative  scaffold  was  delivered  to  the  trainee  via 
a  human  tutor. 

o  Handout:  The  error  preventative  scaffold  was  delivered  to  the  trainee  via  a 
handout. 

o  Computer/web:  The  error  preventative  scaffold  was  delivered  to  the  trainee 
electronically  over  the  computer. 

By  clicking  the  “Next”  button  the  attribute  data  is  saved  in  the  database  and  the  user  can 
advance  to  the  third  step  for  defining  a  contrast  for  effect  size  calculations.  If  you  decide  not  to 
enter  the  contrast  moderator  data  click  the  “Cancel  Create  New  Contrast”  hyperlink  at  the  bottom 
left  of  the  page. 
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Define  Contrast  Statistics 


The  third  step  in  adding  a  contrast  for  effect  size  calculations  is  selecting  the  type  of  data  that 
you  will  enter  for  the  effect  size  calculation.  Here,  you  select  the  type  of  statistics  you  have 
available  for  data  entry.  You  first  select  the  type  of  data  available  from:  descriptive  statistics,  t- 
test  results,  or  F-test  results  by  clicking  the  appropriate  radio  button  (Figure  54).  Once  you  select 
one  of  the  radio  buttons,  TARGET  will  display  additional  options  based  on  the  selection. 


Document  Title:  TEST 

Author(s):  Adair,  D. 

Yean  2013 

Report  Type:  Experimental  Data 

Create  New  Contrast:  Statistics  Options 
^  New  Contrast:  twert  ^ 

*  What  type  of  data  would  you  like  to  input? 

Descriptive  Statistics 
OT-test 
p  F-test 


Next 


*  Required 

Cancel  Create  New  Contrast 


Figure  54.  Create  New  Contrast:  Statistics  Options 
Descriptive  Statistics 

Descriptive  statistics  are  summary  statistics  about  the  sample  and  measures.  Examples  of 
descriptive  statistics  are  the  mean,  standard  deviation,  and  sample  size.  If  you  have  descriptive 
statistics,  one  of  the  following  four  combinations  of  descriptive  statistics  must  be  selected  using 
the  radio  buttons  on  the  page  (Figure  55): 

•  Treatment  mean  (Mt),  Treatment  standard  deviation  (SDt),  Treatment  sample  size  (nt), 
Control  mean  (Mc),  Control  standard  deviation  (SDC),  and  the  Control  sample  size 
(nc). 

•  Treatment  mean  (Mt),  Treatment  standard  deviation  (SDt),  Control  mean  (Mc), 

Control  standard  deviation  (SDC),  and  the  Overall  sample  size  (N),  when  nt  =  nc. 

•  Treatment  mean  (Mt),  Treatment  sample  size  (nt),  Control  mean  (Mc),  the  Control 
sample  size  (nc),  and  the  Pooled  standard  deviation  (SDpO0ied). 

•  Treatment  mean  (Mt),  Control  mean  (Mc),  the  Pooled  standard  deviation  (SDpooied), 
and  the  Overall  sample  size  (N),  when  nt  =  nc. 
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Create  New  Contrast:  Statistics  Options 
^  New  Contrast:  twert  j 


What  type  of  data  would  you  like  to  input? 

*  Descriptive  Statistics 
T-test 
F-test 

*  Choose  specific  input  for  for  DESCRIPTIVE  STATISTICS 


M,  SD,  and  n  for  Treatment  and  Control 

M,  SD,  and  N  for  Treatment  and  Control  when  sample  sizes  are  equal 
M  and  n  for  Treatment  and  Control,  SD  pooled 

M  for  Treatment  and  Control,  N,  and  SD  pooled  when  sample  sizes  are  equal 


Key 

M  =  Mean 

SD  =  Standard  Deviation 
n  =  Group  sample  size 
N  =  Total  Number  of  participants 
df  =  degrees  of  freedom 


*  Required 

Cancel  Create  New  Contrast 


Figure  55.  Descriptive  Statistics  Options 
T-Test  Results 

T-tests  are  a  class  of  inferential  statistical  tests,  specifically,  the  Student’s  t-test  for  small 
samples.  Examples  of  t-test  output  are  the  t  value,  the  p  value,  the  degrees  of  freedom,  and 
sample  size.  If  you  have  t-test  results,  one  of  the  following  six  combinations  of  t-test  output 
statistics  must  be  selected  using  the  radio  buttons  on  the  page  (Figure  56): 

•  t  value,  Treatment  sample  size  (nt),  and  the  Control  sample  size  (nc). 

•  t  value  and  degrees  of  freedom  (df),  when  nt  =  nc. 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 
dependent  samples  when  entering  data  in  the  next  window.  Dependent 
samples  compare  performance  for  the  same  sample  of  participants  in  both 
training  conditions.  Independent  samples  compare  performance  of  different 
samples  of  participants,  one  for  each  condition. 

•  t  value  and  the  Overall  sample  size  (N),  when  nt  =  nc. 

•  p  value  of  t-test,  Treatment  sample  size  (nt),  and  the  Control  sample  size  (nc). 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  above  for  the 
distinction). 

•  p  value  of  t-test  and  degrees  of  freedom  (df),  when  nt  =  nc. 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  above  for  the 
distinction). 
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•  p  value  of  t-test  and  the  Overall  sample  size  (N),  when  nt  =  nc. 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  above  for  the 
distinction). 


Create  New  Contrast:  Statistics  Options 

^  New  Contrast:  twert  j 

*  What  type  of  data  would  you  like  to  input? 

Descriptive  Statistics 
'••T-test 
OF- test 

*  Choose  specific  input  for  t-TEST 

t  value  and  n  for  Treatment  and  Control 
t  value  and  df  when  sample  sizes  are  equal 
t  value  and  N  when  sample  sizes  are  equal 
t-test  p  value,  n  for  Treatment  and  Control 
t-test  p  value  and  df  when  sample  sizes  are  equal 
t-test  p  value  and  N  when  sample  sizes  are  equal 


Key 

M  =  Mean 

SD  =  Standard  Deviation 
n  =  Group  sample  size 
N  =  Total  Number  of  participants 
df  =  degrees  of  freedom 


Next 


*  Required 

Cancel  Create  New  Contrast 

Figure  56.  t-Test  Statistics  Options 

F-Test  Results 

F-tests  are  a  class  of  inferential  statistical  tests,  often  run  in  association  with  analysis  of 
variance.  Examples  of  F-test  output  are  the  F  value,  the  p  value,  the  degrees  of  freedom 
numerator,  the  degrees  of  freedom  denominator,  and  sample  size.  If  you  have  F-test  results,  one 
of  the  following  six  combinations  of  F-test  output  statistics  must  be  selected  using  the  radio 
buttons  on  the  page  (Figure  57): 

•  F  value,  Treatment  sample  size  (nt),  and  the  Control  sample  size  (nc). 

•  F  value  and  degrees  of  freedom  (df),  when  nt  =  nc. 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  t-test  results 
section  above  for  the  distinction). 

•  F  value  and  the  Overall  sample  size  (N),  when  nt  =  nc. 

•  p  value  of  F-test,  Treatment  sample  size  (nt),  and  the  Control  sample  size  (nc). 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  above  for  the 
distinction). 
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•  p  value  of  F-test  and  degrees  of  freedom  (df),  when  nt  =  nc. 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  above  for  the 
distinction). 

•  p  value  of  F-test  and  the  Overall  sample  size  (N),  when  nt  =  nc. 

o  For  this  option,  the  sample  must  be  defined  as  independent  samples  or 

dependent  samples  when  entering  data  in  the  next  window  (see  above  for  the 
distinction). 


Create  New  Contrast:  Statistics  Options 

■  New  Contrast:  twert  1 

*  What  type  of  data  would  you  like  to  input? 

Descriptive  Statistics 
OT-test 
®  F-test 

*  Choose  specific  input  for  F-TEST 

F  value  and  n  for  Treatment  and  Control 
F  value  and  df  when  sample  sizes  are  equal 
F  value  and  N  when  sample  sizes  are  equal 
F-test  p  value,  n  for  Treatment  and  Control 
F-test  p  value  and  df  when  sample  sizes  are  equal 
F-test  p  value  and  N  when  sample  sizes  are  equal 


Key 

M  =  Mean 

SD  =  Standard  Deviation 
n  =  Group  sample  size 
N  =  Total  Number  of  participants 
df  =  degrees  of  freedom 


Next 


*  Required 

Cancel  Create  New  Contrast 

Figure  57.  F-Test  Statistics  Options 

By  clicking  the  “Next”  button  you  will  be  taken  to  another  window  where  you  can  enter  your 
data  for  effect  size  computation.  Entering  user  data  is  the  last  step  in  entering  contrast 
information  for  the  effect  size  calculation. 
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Data  Entry  and  Computing  the  Effect  Size 

The  fourth  and  final  step  in  entering  contrast  information  for  the  effect  size  calculation  is 
data  entry.  Here,  a  number  of  fields  are  available  to  you  that  are  specific  to  the  data  type 
information  defined  in  the  previous  step.  An  example  of  the  data  entry  fields  for  a  set  of 
descriptive  statistics  is  shown  in  Figure  58,  for  a  set  of  t-test  results  in  Figure  59,  and  for  a  set  of 
F-test  results  in  Figure  60. 


Create  New  Contast:  Descriptive  Statistics,  Data  Entry 

|  New  Contrast:  twertj 


Treatment  group  Control  group 


Mean  (M): 

tt 

Mean  (M) 

!±1 

Standard  deviation  (SD): 

tt 

Standard  deviation  (SD) 

m 

Sample  size  (n): 

l±! 

Sample  size  (n) 

_ a 

Direction  Of  The  Effect  p 

|  Select  a  Direction  Of  The  Effect  |  ▼  | 


Next 


Cancel  Create  New  Contrast 


Figure  58.  Example  Descriptive  Statistics  Entry 


Figure  59.  Example  t-Test  Data  Entry 


Create  New  Contast: ,  Data  Entry 
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Treatment  group  Control  group 

Sample  size  (n):|  ~jT|  Sample  size  (n):|  "El 

Direction  Of  The  Effect  i 

|  Select  a  Direction  Of  The  Effect  |T~| 


Next 


Cancel  Create  New  Contrast 


New  Contrast:  twert 

f-value:| 


Figure  60.  Example  F-Test  Data  Entry 
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The  last  piece  of  information  you  must  provide  before  computing  the  effect  size  is  the 
direction  of  effect.  The  direction  of  effect  is  indicated  with  a  drop-down  as  shown  in  the  figures 
above. 

•  Effect  size  describes  the  magnitude  of  the  difference  between  the  treatment  training 
condition  and  control  condition;  however,  it  also  describes  the  direction  of  the  effect. 

•  A  difference  is  in  the  positive  direction  if  the  treatment  (e.g.,  high  learner  control 
training  group)  shows  a  training  improvement  over  the  control  group  (e.g.,  no  learner 
control  group). 

•  A  difference  is  in  the  negative  direction  if  participants  in  the  treatment  group  perform 
worse  on  average  due  to  the  training  than  participants  in  the  control  group. 

•  If  both  treatment  and  control  groups  were  exactly  the  same,  then  the  difference,  and 
therefore  effect  size,  would  be  zero  (neither  positive  or  negative  in  direction). 

Once  all  data  has  been  entered  and  the  direction  of  effect  has  been  specified,  select  the 
“Next”  button  and  a  set  of  effect  size  statistics  is  presented  for  the  selected  contrast  at  the  bottom 
of  the  window  (Figure  61).  The  top  of  the  window  show  the  statistics  parameters  entered. 


Create  New  Contrast:  Calculated  Statistics 

New  Contrast:  twert 

M  -  treatment  group  10.00 
M  -  control  group  12.00 
SD  -  treatment  group  100 
SD  -  control  group  0.40 
N  -  treatment  group  12.00 
N  -  control  group  20.00 
Direction  Of  The  Effect  Positive 


Hedges'  g  p 

2.85 

Variance  p 

0.25 

Standard  Error  p 

0.50 

95th  Upper  Limit  p 

3.82 

95th  Lower  Limit  p 

1.88 

Add  Contrast 

Figure  61.  Calculated  Statistics  Example 
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Once  the  effect  size  statistics  have  been  computed,  you  need  to  click  on  the  “Add  Contrast” 
button.  TARGET  will  save  all  the  contrast  parameters  to  the  database.  If  the  contrast  has  been 
successfully  added,  TARGET  will  display  a  page  showing  that  the  contrast  was  created 
successfully  (Figure  62). 


Create 

r 

Create 

Contrast  twert  created  successfully. 

Finished  Visualize  Effects 

_  1 

Figure  62.  Contrast  Created  Successfully 


From  here,  you  can  either  click  the  “Finished”  button  to  go  back  to  the  contrast  index  page  or 
you  may  link  to  the  forest  plots  under  Explore  Methods  for  the  training  method  coded  under  the 
current  contrast.  To  link  to  a  forest  plot  for  the  training  method  of  the  new  contrast,  click  to  the 
“Visualize  Effects”  button  at  the  bottom  of  the  window.  TARGET  will  take  you  to  the  Explore 
Methods  page  and  display  the  forest  plat  data  for  the  training  method  selected.  The  “Include 
New  Studies”  checkbox  under  “Advanced  Options”  will  be  automatically  checked  so  that  the 
effects  from  the  newly  entered  contrast  are  used  in  the  effect  size  calculations. 


Authors  Submenu 


Selecting  the  “Authors”  submenu  under  the  “Documents”  menu  will  open  the  Authors  Index 
page  (Figure  63).  From  here,  you  can  view/edit  the  list  of  all  the  authors  in  the  current  database 
and  view  a  list  of  documents  for  any  selected  author. 
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Index 


Create  New 


Last  Name 

First  Name 

Abbot 

R.  G. 

Edit 

1  Details 

1  Delete 

Abrami 

P.  C. 

Edit 

1  Details 

|  Delete 

Adair 

D. 

Edit 

1  Details 

|  Delete 

Adams 

D.  M. 

Edit 

1  Details 

|  Delete 

Adams 

J.A. 

Edit 

1  Details 

1  Delete 

Ad  apathy  a 

R.  S. 

Edit 

1  Details 

1  Delete 

Aggarwal 

R. 

Edit 

|  Details 

|  Delete 

Aguinis 

H. 

Edit 

|  Details 

j  Delete 

Ahmad 

S.  W. 

Edit 

1  Details 

1  Delete 

Aldrich 

N.J. 

Edit 

1  Details 

1  Delete 

Aleven 

V. 

Edit 

1  Details 

|  Delete 

Alfieri 

L. 

Edit 

1  Details 

|  Delete 

Alge 

B.J. 

Edit 

|  Details 

1  Delete 

Allen 

G.W. 

Edit 

1  Details 

1  Delete 

Alliger 

G.  M. 

Edit 

1  Details 

1  Delete 

Alonso 

D.  L. 

Edit 

1  Details 

|  Delete 

Alseidi 

A.  A. 

Edit 

1  Details 

|  Delete 

Alvarez 

K. 

Edit 

1  Details 

1  Delete 

Aly 

M. 

Edit 

1  Details 

1  Delete 

Anderson 

J.  R. 

Edit 

1  Details 

j  Delete 

Anderson 

T. 

Edit 

1  Details 

|  Delete 

M  r _ 

Figure  63.  Author  Index 

You  can  select  any  of  the  options  “Edit,”  “Details,”  or  “Delete”  to  the  right  of  each  author 
name.  As  a  contributor  you  can  only  edit  or  delete  authors  that  you  added.  The  “Edit”  option 
will  open  the  Edit  page  (Figure  64)  where  you  can  view/edit  the  author’s  first  and  last  name.  If 
you  make  any  changes,  click  on  the  “Save”  button  to  commit  your  changes. 


Edit 

Author 


Last  Name 


[Abbot 

1 

First  Name 

|R.  G. 

1 

Save 

Back  to  List 


Figure  64.  Edit  Author  Page 
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The  “Details”  option  will  show  you  the  author  information  and  a  list  of  documents  associated 
with  that  author  (Figure  65). 


Details 


Last  Name 

McDermott 

First  Name 

Patricia  L 


Documents 


Document  Title 

Author(s) 

Year 

Data 

Source 

Report  Type 

Date 

Created 

Transferring  from  the  simulator  to  a  live  robotic  environment:  The  effectiveness  of  part-task  and  whole-task  training.  Submitted 
for  consideration  as  a  Technical  Report  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  Fort  Belvoir,  VA. 

McDermott  Patricia  L:  Carolan,  Thomas  F.; 
Gronowski.  Mark  It;  Gacy,  Marc  Fisher. 

Alia 

2012 

Experimental 

Data 

2013 

Edit  | 
Details 

The  Impact  of  Adaptive  Training  on  Transfer.  Report  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 
[Experiment  4] 

McDermott  Patricia  L,-  Carolan.  Thomas  Fj 
Gronowski.  Mark  Rj  Gacy.  Marc  Fisher. 

Alia 

2012 

Experimental 

Data 

2013 

Ml  1 

Details 

The  impact  of  adaptive  training  on  transfer.  Report  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 
[Experiment  5,  Error  Prevention] 

McDermott  Patricia  L:  Carolan,  Thomas  F.; 
Gronowski,  Mark  It;  Gacy,  Marc  Fisher. 

Alia 

2012 

Experimental 

Data 

2013 

Edit  | 
Details 

Transfer  effectiveness  of  exploration-based  learning  for  digital  skills.  Submitted  for  consideration  as  a  Technical  Report  to  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences.  Fort  Belvoir.  VA. 

McDermott  Patricia  U  Carolan,  Thomas  F^ 
Gronowski.  Mark  Rj  Fisher.  Alia 

2011 

<No 

Source 

Defined* 

Experimental 

Data 

2013 

Ml  1 
Details 

The  usefulness  of  worked  examples  as  a  training  aid  in  the  context  of  unmanned  vehicle  route  planning.  Submitted  for 
consideration  as  a  Technical  Report  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  Fort  Belvoir.  VA. 

McDermott  Patricia  L:  Carolan.  Thomas  F.; 
Gronowski.  Mark  It:  Gacy.  Marc 

2012 

Experimental 

Data 

2013 

Ml  | 
Details 

Edit  |  Back  to  Ust 

Figure  65.  Author  Details  Page 


The  “Delete”  option  will  cause  TARGET  to  delete  that  author  from  the  database. 

Data  Sources  Submenu 

Selecting  the  “Data  Source”  submenu  under  the  “Documents”  menu  will  open  the  Data 
Source  Index  page  (Figure  66).  From  here,  you  can  view/edit  the  list  of  all  the  data  sources  in 
the  current  database. 


Index 

Create  New 
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Academic  Medicine 
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Edit  |  Details  |  Delete 

American  Educational  Research  Journal 

Edit  |  Details  |  Delete 

American  Institute  of  Physics 

Edit  |  Details  |  Delete 

American  Journal  of  Distance  Education 

Edit  |  Details  |  Delete 

American  Journal  of  Psychology 

Edit  |  Details  1  Delete 

Anesthesiology 

Edit  |  Details  1  Delete 

Annual  Review  of  Psychology 

Edit  |  Details  |  Delete 

Figure  66.  Data  Source  Index  Page 
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You  can  select  any  of  the  options  “Edit,”  “Details,”  or  “Delete”  to  the  right  of  each  data 
source  name.  As  a  contributor  you  can  only  edit  or  delete  data  sources  that  you  added.  The 
“Edit”  option  will  open  the  Edit  page  where  you  can  view/edit  the  name  of  the  data  source.  If 
you  make  any  changes,  click  on  the  “Save”  button  to  commit  your  changes. 

The  “Details”  option  will  show  you  the  data  source  infonnation  and  the  “Delete”  option  will 
remove  the  data  source  from  the  database. 
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Help  Menu 


View  Training  Method  Definitions 

The  view  Training  Method  Definitions  function  allows  users  to  view  extended  descriptions 
of  different  training  methods,  such  as  Part-Task  training.  To  view  a  training  method  definition, 
select  “Definitions”  from  the  “Help”  menu  located  at  the  top  of  the  Explore  Documents  window 
(Figure  67). 


Home  Tasks  Methods  Documents  Help  About  Admin 

Definitions 

Figure  67.  Definitions  Submenu 

Once  selected,  a  range  of  training  method  options  will  be  displayed  in  a  drop-down  list. 
Selecting  a  method  will  open  a  description  and  summary  of  the  selected  training  method 
(Figure  68). 


jnal  method  that  provides  a  learner  with  control  over  learning  decisions  made  during  the  course  of  training  (Carolan,  Hutchins,  &  Wickens,  2012).  There  are  typically  four  dimensions  of 
control  oven  (1)  branching  -  what  material  is  viewed,  (2)  pace  -  how  long  material  is  viewed,  (3)  truncation  of  lesson  -  when  to  stop  the  training  session,  and  (4)  frequency  of  feedback  or 
practice  (Kraiger,  Jerden,  2007).  The  amount  of  control  given  to  a  trainee  over  these  four  dimensions  varies. 

•  Programmatic  control  -  The  trainee  has  no  control  over  the  four  dimensions  (branching,  pacing,  lesson  truncation,  or  feedback  truncation). 

•  Partial  learner  control  -  The  trainee  has  control  over  only  select  dimensions. 

•  Full  learner  control  -  The  trainee  has  control  over  all  four  dimensions. 

Three  training  design  questions  specific  to  learner  control  deal  with  the  use  of  feedback,  practice  and  advisement  (Carolan,  Hutchins,  &  Wickens,  submitted). 

•  Feedback  -  Does  providing  performance  feedback  after  each  module  or  a  group  of  modules  increase  the  effectiveness  of  the  training  strategy? 

•  Practice  -  Does  practice  or  quizzes  at  pre-determined  point  in  training  increase  the  effectiveness  of  the  training  strategy? 

•  Advisement  -  Does  the  inclusion  of  an  intelligent  agent  which  recommends  pacing  and  branching  options  based  on  trainee  performance  increase  the  effectiveness  of  the  training  strategy? 

In  the  research  that  was  reviewed  learner  control  was  typically  used  as  a  strategy  for  training  cognitive  skills  and  tasks,  there  were  no  experiments  involving  perceptual  or  psychomotor  tasks.  While  a  majority  of  the 
experimental  findings  were  for  the  training  of  declarative  knowledge  other  cognitive  skills  were  represented;  such  as  problem-solving,  procedural  knowledge  and  quantitative  reasoning.  The  experimental  research 
focused  on  computer-based  or  web-based  training  systems,  with  no  instructor  available  to  provide  additional  support  or  instruction. 

Learner  Control  Moderators 

Control  vs.  Treatment  Contrast 

In  the  Learner  Control  (LC)  research  there  are  four  dimensions  of  control  given  to  the  trainee:  branching,  pacing,  lesson  truncation  and  feedback/practice  truncation.  Training  conditions  must  be  coded  based  on  the 
number  and  type  of  dimensions  of  control  given  to  the  participant.  There  are  three  categories  a  learner  control  experiment  condition  can  be  placed  into:  programmatic  control  (i.e.  control  condition,  no  learner  control), 
partial  learner  control,  and  full  learner  control.  There  are  four  possible  comparisons  between  control  dimensions:  (1)  programmatic  control  vs.  full  learner  control,  (2)  programmatic  control  vs.  partial  learner  control,  (3) 
partial  learner  control  vs.  full  learner  control,  and  (4)  partial  learner  control  with  more  dimensions  vs.  partial  learner  control  with  fewer  dimensions. 

1.  Programmatic  vs.  Full  LC  -  The  experiment  compares  a  control  group  that  had  no  control  over  their  training  (programmatic  control)  and  a  treatment  group  that  had  control  over  all  four  dimensions  of  training 
(full  learner  control). 

2.  Programmatic  vs.  Partial  LC- The  experiment  compares  a  control  group  that  had  no  control  over  their  training  (programmatic  control)  and  a  treatment  group  that  had  control  over  on^  two  or  three  dimensions 
of  training  (partial  learner  control). 

3.  Partial  LC  vs.  Full  LC-The  experiment  compares  a  control  group  that  had  control  over  one,  two  or  three  dimensions  of  training  (partial  learner  control)  and  a  treatment  group  that  had  control  over  all  four 
dimensions  of  training  (full  learner  control). 

4.  Partial  Min  vs.  Partial  Max -The  experiment  compares  a  control  group  that  had  control  over  fewer  dimensions  than  the  treatment  group.  For  both  groups  an  intermediate  number  of  dimensions  were  under 
learner  control,  not  all  four  dimensions. 

Control  and  Treatment  Condition 

For  each  contrast,  the  specific  dimensions  being  manipulated  consisted  primarily  of  control  oven  branching,  pace,  truncation  of  lessons,  and  truncation  of  reinforcing  material  such  as  feedback  or  practice.  This 
moderator  indicates  which  of  the  four  dimensions  were  manipulated  in  the  experimental  contrast. 

1.  Pace-Trainee  was  able  to  control  the  amount  of  time  he  spent  viewing  a  specific  screen,  item,  or  lesson. 

2.  Branching -Trainee  was  able  to  control  what  material  he  viewed  and  the  order  in  which  he  viewed  it. 

3.  Lesson  Truncation -Trainee  was  able  to  stop  viewing  particular  portions  of  the  training  material. 

4.  Feedback/Practice  Truncation  -Trainee  was  able  to  control  whether  heviewed  or  received  feedback  and/or  how  much  practice  he  did  priorto  moving  onto  the  next  portion  of  the  training  material. 

5.  Multiple -Trainee  was  able  to  control  more  than  one  of  the  following  training  dimensions:  pacing,  branching,  lesson  truncation,  and  feedback/practice  truncation. 

6.  Other -Trainee  was  able  to  control  an  aspect  of  the  training  system  or  material  that  is  not  fit  into  one  of  the  following  categories:  pacing,  branching,  lesson  truncation  and  feedback/practice  truncation 


Figure  68.  Definitions  Page 
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Admin  Menu 


The  Admin  menu  option  is  only  visible  to  users  with  Administrator  privileges.  The  Admin 
menu  offers  three  submenu  choices:  Categories,  Attributes,  or  Document  Summary  MetaData. 
Each  of  these  are  described  in  this  section 

Admin  Categories  Menu 

Selecting  the  “Categories”  option  under  the  “Admin”  menu  will  display  a  window  similar  to 
Figure  69.  This  page  allows  you  to  view/edit  the  categories  that  are  displayed  on  the  Document 
attributes  page. 


Index 

Create  New 


Name  Description  ColumnPosition  Ordinal 


Sample 

3 

2 

Edit 

|  Details 

1  Delete 

Delivery  Method 

3 

3 

Edit 

|  Details 

1  Delete 

Training  Method:  Other 

1 

2 

Edit 

|  Details 

1  Delete 

Task  Category 

2 

1 

Edit 

|  Details 

1  Delete 

Task  Type:  Military  Specific 

2 

1 

Edit 

|  Details 

1  Delete 

Individual  Factors 

3 

1 

Edit 

1  Details 

1  Delete 

Outcome  Measure 

3 

1 

Edit 

|  Details 

1  Delete 

Meta-analysis 

3 

1 

Edit 

|  Details 

1  Delete 

Training  Method:  Practice 

1 

1 

Edit 

|  Details 

1  Delete 

Training  Method:  Managing  Difficulty 

1 

1 

Edit 

|  Details 

1  Delete 

Task  Type:  Interpersonal  Skills 

2 

2 

Edit 

|  Details 

1  Delete 

Task  Type:  Miscellaneous 

2 

3 

Edit 

|  Details 

1  Delete 

Figure  69.  Admin  Categories  Page 
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You  can  select  any  of  the  options  “Edit,”  “Details,”  or  “Delete”  to  the  right  of  each  category. 
The  “Edit”  option  will  open  the  Edit  page  (Figure  70)  where  you  can  view/edit  the  category 
name,  description,  column  position  and/or  ordinal  value.  If  you  make  any  changes,  click  on  the 
“Save”  button  to  commit  your  changes. 


Edit 


Name 

|Sample 

1 

Description 

ColumnPosition 

|3 

m 

Ordinal 
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a 

Save 

Back  to  List 


Figure  70.  Edit  Categories  Window 

The  “Details”  option  will  show  you  the  category  information  but  you  cannot  edit  it. 


The  “Delete”  option  will  cause  TARGET  to  delete  that  category.  All  attributes  defined  under 
that  category  will  also  be  deleted. 


Admin  Attributes  Menu 


Selection  of  the  “Attributes”  option  under  the  “Admin”  menu  displays  the  Attributes  Index 
page  (Figure  71).  From  here  you  can  view/edit  the  attributes  that  are  displayed  under  each 
category  on  the  Document  attributes  page. 


Index 

Create  New 

Category 

Name 

Description 

Ordinal 

Sample 

Work 

Study  sample  is  people  in  the  workforce. 

3 

Edit  | 
Details 

| 

Sample 

Military 

Study  sample  is  military  personnel. 

2 

Delete 
Edit  | 
Details 

| 

Sample 

Academic 

Study  sample  is  undergraduate  or  high  school  students. 

1 

Delete 
Edit  | 
Details 

Delivery 

Classroom 

The  primary  delivery  environment  is  classroom-based  led  by  an  instructor  and  may  include  the  use  of  lecture  computers,  handouts,  and/or  texts. 

1 

Delete 
Edit  | 
Details 

Delivery 

Computer- 

The  primary  delivery  environment  is  a  stand-alone  computer-based  training  system  that  is  used  to  present  content  and  assess  students'  learning.  (Keyword:  company 

Delete 
Edit  | 
Details 

1 _ 

Figure  71.  Admin  Attributes  Index  Page 
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You  can  select  any  of  the  options  “Edit,”  “Details,”  or  “Delete”  to  the  right  of  each  attribute. 
The  “Edit”  option  will  open  the  Edit  page  (Figure  72)  where  you  can  view/edit  the  category  the 
attribute  belongs  to,  attribute  name,  description,  and/or  ordinal  value.  If  you  make  any  changes, 
click  on  the  “Save”  button  to  commit  your  changes. 


Figure  72,  Attributes  Edit  Page 

The  “Details”  option  will  show  you  the  attribute  infonnation  but  you  cannot  edit  it. 

The  “Delete”  option  will  cause  TARGET  to  delete  that  attribute. 

Admin  Document  Summary  MetaData  Menu 

The  “Document  Summary  MetaData”  option  allows  you  to  view/edit  the  document  fields  and 
field  definitions  that  appear  on  the  Document  Summary  page.  If  you  select  the  “Document 
Summary  MetaData”  submenu  under  the  “Admin”  menu,  TARGET  will  display  the  MetaData 
Index  page  (Figure  73).  From  here  you  can  view/edit  the  data  fields  and  definitions  that  are 
displayed  on  the  Document  Summary  page. 


Index 

Column  Name 

Arti  c  I  eD  esc  ri  pti  o  n 

TrainingMethodDefin'ition 

TaskSkillDescription 


Experimental  Training  Method 
Data  Definition 

Exp  eri  m  enta  I  Ta  rg  et  Ta  sk/Ski  1 1 
Data  Description 


Description 

A  brief  description  of  the  article  what  is  being  described,  what  training  methods  or  skills  were  discussed  in  the  report. 
What  training  methods  are  investigated  in  the  report  and  how  they  are  define  by  the  authors. 

Description  of  the  task  type  or  skill  areas  the  report  addresses;  e.g.  perceptual,  cognitive  mathematics,  and  aviation. 


TheoreticalConce  tualFramework  Experimental  Theoretical/Conceptual  A  summary  of  the  theory,  methodology,  framework,  or  hypotheses  that  the  author(s)  use  to  organize  the  research,  review,  meta-analysis 
^  Data  Approach  or  tool  development  described  in  the  report. 


StudyCriteria 

Stu  dy  P  a  rticip  a  nts 

OperationalDefinition 

Shn-h.nitinn _ 


Experimental ,.  ,  r  „ . 
nJl,  Study  Setting 


Data 


Environment  in  which  study  was  conducted:  e.g.  military,  academic  or  work. 


Experimental  <^ujy  participants  Description  of  the  participants;  e.g.  relevant  demographics,  group  differences,  and  sample  i 


Data 

Experimental  Training  Task 
Data  Description 

Experimental  Cl.  ^  w  n„: _ 


Detailed  description  of  the  experimental  task(s). 

Description  of  the  type  of  e<periment  or  evaluation  and  its  overall  design;  e.g.  independent  variables,  number  of  levels  for  each  variable. 


Edit  | 
Details 
Edit  | 
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Edit  | 
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Edit  | 
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Edit  | 
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Edit  | 
Details 
Edit  | 
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Edit  I 


Figure  73.  Admin  Document  MetaData  Page 
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You  can  select  “Edit”  or  “Details”  to  the  right  of  each  attribute.  The  “Edit”  option  will  open 
the  Edit  page  (Figure  74)  where  you  can  view/edit  the  Report  Type,  display  name,  and/or 
description.  If  you  make  any  changes,  click  on  the  “Save”  button  to  commit  your  changes. 


Edit 

DocumentSummaryMetaData 

Column  Name 

StudyCriteria 

ReportType 

Experimental  Data  pY 

Display  Name 

[study  Setting  | 

Description 

I  Environment  in  which  study  was  conducted:  e.g.  military,  academic  or  work. 


|  Save 

Back  to  List 


Figure  74.  Edit  MetaData  Page 

The  “Details”  option  will  show  you  the  metadata  infonnation  but  you  cannot  edit  it. 
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Troubleshooting 


This  section  details  known  issues  and  presents  solutions. 

1.  “On  the  Explore  Task  page  I  cannot  see  the  bubble-chart.  Only  a  blank  square 
is  shown  on  the  page.” 

a.  If  you  are  using  Internet  Explorer  make  sure  that  compatibility  mode  is  off. 

To  check  if  compatibility  mode  is  on/off  click  on  the  Tools  menu.  If  there  is  a 
checkmark  next  to  “Compatibility  View”  then  it  is  on.  To  turn  it  off  click  on 
the  “Compatibility  View”  menu  item.  If  you  are  using  Internet  Explorer  10 
you  will  need  to  go  to  “Compatibility  View  Settings”  and  uncheck  all  settings. 

2.  “On  the  Explore  Methods  page  I  cannot  see  the  forest  plot.  Only  a  blank  square 
is  shown  on  the  page.” 

a.  If  you  are  using  Internet  Explorer  make  sure  that  compatibility  mode  is  off. 

To  check  if  compatibility  mode  is  on/off  click  on  the  Tools  menu.  If  there  is  a 
checkmark  next  to  “Compatibility  View”  then  it  is  on.  To  turn  it  off  click  on 
the  “Compatibility  View”  menu  item.  If  you  are  using  Internet  Explorer  10 
you  will  need  to  go  to  “Compatibility  View  Settings”  and  uncheck  all  settings. 

3.  “I  get  sent  to  an  unauthorized  page.” 

a.  Contributor  level  users  (default)  cannot  delete  or  edit  any  documents,  authors, 
or  data  sources  that  they  did  not  create.  Attempting  to  edit  or  delete  the 
TARGET  library  data  will  result  in  the  unauthorized  page  being  shown. 

4.  “Server  Error  in  ‘/Target’  Application  message 

a.  If  you  see  this  message  it  means  that  your  session  has  timed  out.  Go  back  to 
the  login  page  at  http://bldr-webtest.alionscience.com/Target/  and  re-login  to 
the  site. 
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Appendix  A  -  Attribute  Definitions 


Category 

Attribute  Name 

Attribute  Definition 

Training  Strategy: 

Practice 

Group  Practice 

Practice  in  pairs  or  small  groups. 
Trainees  can  be  either  co-located, 
working  on  a  task  face-to-face,  or 
distributed,  working  together 
electronically.  (Keywords:  group 
discussion,  small  group  projects) 

Training  Strategy: 

Practice 

Mental  Practice 

Mentally  rehearsing  the  steps  in  a  task. 
(Keywords:  symbolic  mental  rehearsal, 
visualization  exercises) 

Training  Strategy: 

Practice 

Overlearning 

Intentionally  continuing  practice  on  a 
task  after  the  trainee  initially  reaches 
the  skill  level  required  to  be  considered 
proficient  at  the  task.  (Keywords: 
degree  of  overlearning,  repeated 
practice) 

Training  Strategy: 

Practice 

Practice  Spacing 

Time  interval  between  repetitions  of 
practice  trials;  either  no  time  (massed) 
or  some  predetermined  interval 
(spaced).  (Keywords:  spaced  practice, 
massed  practice) 

Training  Strategy: 

Practice 

Scenario-Based  Practice 

Scenario-based  practice,  which  is 
learner-centric  and  requires  the  trainee 
to  leam  by  doing.  Trainee  encounters 
cues  and  decisions  similar  to  those  he 
would  be  expected  to  deal  with  in  a 
real  mission  or  on-the-job.  Usually  in  a 
synthetic  learning  environment. 
(Keywords:  theme-based,  goal-based 
scenarios) 

Training  Strategy: 
Managing  Difficulty 

Adaptive  Training 

Task  difficulty  is  automatically  varied, 
or  adapted,  dependent  upon  trainee 
performance  in  order  to  maintain  a 
constant  level  of  performance  (i.e., 
effort  demand)  as  trainee  learning  of  a 
task  proceeds.  (Keywords:  intelligent 
tutoring,  adaptive  remediation) 
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Category 

Attribute  Name 

Attribute  Definition 

Training  Strategy: 
Managing  Difficulty 

Exploration 

A  trainee  explores  a  task  environment 
on  his  own.  The  level  or  type  of 
guidance  given  to  the  trainee  vary  (i.e., 
only  a  user  manual  vs.  conceptual  input 
when  trainer  queried).  (Keywords: 
discovery  learning,  error  management 
training) 

Training  Strategy: 
Managing  Difficulty 

Increasing  Difficulty 

Parameters  of  the  target  whole  task  are 
initially  set  to  lower  levels,  to  reduce 
the  intrinsic  load  during  early  training 
trials,  and  then  increased  as  training 
progresses,  until  the  difficulty  reaches 
the  level  of  the  target  task.  The 
difficulty  levels  can  increase  in  either  a 
fixed  or  adaptive  schedule.  (Keywords: 
part-to-whole  training,  simplification) 

Training  Strategy: 
Managing  Difficulty 

Learner  Control 

An  instructional  method  that  provides 
trainees  with  decision  making  control 
over  specific  dimensions  or  activities 
within  a  structured  learning 
environment.  (Keywords:  pace  control, 
open  access) 

Training  Strategy: 
Managing  Difficulty 

Part  Task  Training 

Training  method  that  decomposes 
complex  tasks  into  a  series  of  smaller 
tasks,  each  of  which  is  demonstrated 
and  practiced  separately  before  being 
practiced  as  a  whole  task.  (Keywords: 
segmentation,  simple-to-complex) 

Training  Strategy: 
Managing  Difficulty 

Scaffolding 

Assistive  supports  are  provided  to  the 
trainee  to  ease  the  demands  of 
independent  task  performance.  The 
scaffold  supports  are  incrementally 
faded  out  over  time  until  the  trainee  is 
executing  the  whole  task  independently 
with  minimal  demand.  (Keywords: 
prompts,  fading) 
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Category 

Attribute  Name 

Attribute  Definition 

Training  Strategy: 
Managing  Difficulty 

Training  Wheels/Error 
Prevention 

An  error  prevention  strategy  geared 
towards  reducing  the  complexity  of  the 
transfer  task  during  initial  learning  for 
the  purpose  of  reducing  training  task 
errors,  and  to  help  trainees  acquire  the 
appropriate  schema  to  assimilate  the 
transfer  task  throughout  learning. 
(Keywords:  prompts,  lockouts) 

Training  Strategy: 
Managing  Difficulty 

Variable  Priority  Training 

Training  in  which  different 
components  of  whole  task  are 
systematically  emphasized  or 
deemphasized  to  allow  more  attention 
to  be  focused  on  the  former  while  still 
preserving  the  necessary  element  of 
time  sharing.  (Keywords:  changing 
priority,  integrated  approach) 

Training  Strategy: 
Managing  Difficulty 

Worked  Examples 

Training  resembling  training  wheels 
and  scaffolding,  but  differs  in  how 
much  of  the  target  task  is  initially 
facilitated  for  the  trainee.  In  training 
problem  solving,  problems  early  in  the 
learning  sequence  have  some,  or  even 
all  of  the  steps  carried  out  for  the 
learner.  (Keywords:  solution  only, 
decreasing  support) 

Training  Strategy:  Other 

Behavior  Modeling 

Observing  and  reproducing  a  sequence 
of  new  behaviors.  Trainee  leams  by 
watching  another  person  engage  in  a 
sequence  of  behaviors.  (Keywords: 
guided  demonstration,  role-playing.) 

Training  Strategy:  Other 

Feedback 

Providing  infonnation  to  trainees  about 
their  progress  in  order  to  reduce  the 
gap  between  current  and  desired 
performance.  The  guidance  can  target 
either  outcome-related  performance 
(knowledge  of  results)  or  the  process 
of  task  perfonnance  (knowledge  of 
performance).  (Keywords:  directive 
feedback,  fonnative  feedback) 
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Category 

Attribute  Name 

Attribute  Definition 

Training  Strategy:  Other 

Goal  Based 

Trainee  is  required  to  set  a  specific  and 
attainable  goal  prior  to  training. 
(Keywords:  goal  setting,  goal 
attainment) 

Training  Strategy:  Other 

Prior  knowledge  activation 

Using  a  trainees  pre-existing 
knowledge  to  help  with  the  acquisition 
of  new  knowledge.  Trainers  relate  new 
material  to  material  already  known  by 
trainees.  (Keywords:  engaging  prior 
knowledge,  contrasting  cases) 

Training  Strategy:  Other 

Problem  Based  Learning 

A  learner-centric  training  method  that 
involves  trainees  working  together  in 
small  groups  to  solve  a  written 
problem  (case-study).  The  trainer 
provides  a  problem-solving  process 
and  if  lectures  are  provided  they  are 
minimal.  (Keywords:  group  work, 
student-centered) 

Training  Strategy:  Other 

Proceduralized  Instruction 

Trainer  provides  trainee  either  step-by- 
step  instruction  of  how  to  perfonn  a 
task  and/or  instructions  on  the 
conceptual  knowledge  needed  to  learn 
a  task.  Trainees  are  provided  with 
direct  guidance.  (Keywords:  direct 
instructional  guidance,  procedural 
infonnation) 

Training  Strategy:  Other 

Team  Training 

Training  teams  as  a  whole,  instead  as 
individuals.  Training  is  designed  such 
that  trainees  acquire  new  infonnation, 
skills,  and  interpersonal  skills  as  a 
group  with  the  goal  of  improving  the 
team’s  perfonnance.  (Keywords: 
cooperative  learning,  collaborative 
training) 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Category 

Cognitive-Complex 

Decision  Making 

Situation  assessment  and  decision 
making  skills  in  complex,  dynamic  and 
time-sensitive  environments  with 
changing  situations,  attentional 
demands,  application  of  strategies  and 
multiple  goals.  It  may  involve 
coordination  of  perceptual  and  motor 
skills. 

Task  Category 

Cognitive-Declarative 

This  category  includes  all  verbal 
knowledge  categories  including  facts, 
principles,  knowledge  organization, 
and  verbal  or  graphical  mental  models. 

Task  Category 

Cognitive-Problem  Solving 

Application  of  principles,  rules  and 
concepts  to  process  information  and 
solve  problems.  Includes  general 
logical  reasoning  skills;  inductive, 
deductive,  and  diagnostic. 

Task  Category 

Cognitive-Procedural 

Performance  of  basic  procedural  skills, 
where  the  procedure  may  include 
cognitive  steps  and  basic  perceptual 
and  motor  steps  such  as  those  involved 
in  navigating  a  computer  interface. 

Task  Category 

Cognitive-Quantitative 

Reasoning 

Application  of  mathematical  concepts 
and  skills  to  solve  problems. 

Task  Category 

Cognitive-Spatial 

Reasoning 

Visual-spatial  skills  involved  in 
determining  spatial  relationships  with 
respects  to  own  body  orientation  and 
with  representing,  transfonning, 
generating,  and  recalling  symbolic, 
nonlinguistic  infonnation.  (Keywords: 
spatial  perception,  spatial  visualization, 
and  mental  rotation.) 

Task  Category 

Interpersonal 

Social  interaction  skills,  including  but 
not  limited  to  verbal  and  nonverbal 
communication,  persuasion,  building 
and  maintaining  relationships,  and 
managing  conflict. 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Category 

Perceptual 

Skills  involved  in  detecting  and 
interpreting  sensory  information  in 
order  to  gain  awareness  and  support 
performance.  The  ability  to  detect  and 
use  sensory  cues.  Include  some 
perceptual  reasoning  tasks. 

Task  Category 

Psychomotor 

Physical  skills  -  movement, 
coordination,  manipulation,  dexterity, 
strength,  and  speed  -  includes  both  fine 
motor  skills  and  gross  motor  skills. 

Task  Type:  Military 
Specific 

Adaptability 

The  ability  to  respond  effectively  to  a 
new  situation  and,  specifically,  to 
unpredicted  changes;  involves 
cognitive  skills  (intuition  and  critical 
thinking)  and  relational  skills  (self- 
awareness  and  self  regulation). 

Task  Type:  Military 
Specific 

Battlespace  Awareness 

Knowledge  and  understanding  of  the 
operational  area’s  environment, 
factors,  and  conditions,  to  include  the 
status  of  friendly  and  adversary  forces, 
neutrals  and  noncombatants,  weather 
and  terrain,  that  enables  timely, 
relevant,  comprehensive,  and  accurate 
assessments,  in  order  to  successfully 
apply  combat  power,  protect  the  force, 
and/or  complete  the  mission. 

Task  Type:  Military 
Specific 

Command  and  Control 

The  exercise  of  authority  and  direction 
by  a  properly  designated  commander 
over  assigned  and  attached  forces  in 
the  accomplishment  of  the  mission. 
Command  and  control  functions  are 
performed  through  an  arrangement  of 
personnel,  equipment, 
communications,  facilities,  and 
procedures  employed  by  a  commander 
in  planning,  directing,  coordinating, 
and  controlling  forces  and  operations 
in  the  accomplishment  of  the  mission. 
Also  called  C2. 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Type:  Military 
Specific 

Critical  Thinking 

Involves  interpreting,  reasoning, 
assessing,  and  monitoring.  The  ability 
to  analyze  a  battle  situation,  recognize 
its  decisive  elements,  and  devise  a 
workable  solution. 

Task  Type:  Military 
Specific 

Digital  Skills 

Using  military  computer  software, 
typically  digital  command  and  control 
systems,  to  accomplish  missions  on  the 
battlefield.  These  skills  include:  the 
procedural  steps  of  navigating  menu 
systems,  creating  reports,  plans, 
overlaps,  reading  maps, 
communicating  and  collaborating  with 
other  groups  involved  in  the  mission, 
managing  a  high  volume  of  incoming 
data  and  using  this  data  to  visualize  the 
battlefield  and  maintain  high  levels  of 
situation  awareness.  (Examples: 

ABCS,  FBCB2,  MCS,  ASAS, 

AFATDS,  CPOF,  and  GSS) 

Task  Type:  Military 
Specific 

Intelligence  Gathering 

Collection  of  intelligence  on  other 
units  or  forces  by  own  units  or  forces. 
Intelligence  is  the  product  resulting 
from  the  collection,  processing, 
integration,  analysis,  evaluation,  and 
interpretation  of  available  infonnation 
concerning  an  adversary,  foreign 
countries  or  areas. 

Task  Type:  Interpersonal 
Skills 

Active  Listening 

Fistening  attentively,  responsively,  and 
nonjudgmentally  in  a  two-way 
exchange  for  the  purpose  of  meaning¬ 
making  and  understanding;  that  is, 
paying  attention  to  both  what  is  being 
say  as  well  as  what  is  being  done  and 
responding  or  reciprocating 
appropriately  to  questions,  ambiguities, 
and  nonverbal  cues. 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Type:  Interpersonal 
Skills 

Assertive  Communication 

The  ability,  willingness,  and  readiness 
to  appropriately  and  succinctly  express 
emotions,  preferences,  opinions, 
concerns,  or  needs  positively,  directly, 
and  with  self-confidence  without  being 
offensive  or  hostile  during  the  social 
interaction  towards  another  person. 

Task  Type:  Interpersonal 
Skills 

Conflict  Resolution 

The  process  in  which  interpersonal 
communication  is  used  to  allow  two 
conflicted  parties  to  reach  an  amicable 
and  satisfactory  point  of  agreement. 

Task  Type:  Interpersonal 
Skills 

Cultural  Awareness 

Cultural  awareness  is  having  the 
cultural  knowledge  and  cultural  skills 
to  effectively  interact  with  individuals 
from  a  different  cultural  background 
and  to  be  able  to  choose  appropriate 
courses  of  actions,  especially  with 
regards  to  negotiation.  Cultural 
knowledge  includes  knowledge  of 
thoughts,  feelings  and  attitudes  of 
oneself,  and  others  within  the  cultural 
situation.  Cultural  skills  include  the 
verbal  and  nonverbal  communication 
skills. 

Task  Type:  Interpersonal 
Skills 

Leadership  Skills 

The  ability  to  influence  people  by 
providing  purpose,  direction  and 
motivation  while  operating  to 
accomplish  a  shared  objective  in  a 
positive  work  environment. 

Task  Type:  Interpersonal 
Skills 

Negotiation 

A  process  in  which  a  dispute  is 
resolved  or  a  compromise  is  reached 
through  the  use  of  discussion  and/or 
bargaining.  Both  sides  in  the  dispute 
attempt  to  come  to  tenns,  build 
understanding,  achieve  consensus, 
reach  agreement,  fonn  contracts  and 
resolve  conflict. 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Type:  Interpersonal 
Skills 

Nonverbal  Communication 

The  encoding  and  decoding  of  body 
language,  postures,  gestures,  facial 
expressions,  proximity,  and  other 
nonverbal  behaviors/signals  to 
enhance,  reinforce,  modify,  or  replace 
the  verbal  messages  of  an  interpersonal 
communication. 

Task  Type:  Interpersonal 
Skills 

Relationship  Building 

The  ability  to  develop  and  foster 
positive  bonds  with  others. 

Task  Type: 

Miscellaneous 

Academic  Knowledge 

A  college  or  high  school  course 
teaching  a  non  science,  mathematics, 
or  computer  programming  topic. 
Examples  include:  psychology, 
architecture,  second  language,  and 
accounting. 

Task  Type: 

Miscellaneous 

Aviation  Domain 

A  wide  range  of  knowledge  and  skills 
required  to  fly  non-military  and/or 
military  aircraft.  Examples  include: 
flight  maneuvers,  in-flight  procedures, 
time-sharing,  HUD  display 
interpretation,  SAM  detection  and  pilot 
characteristics. 

Task  Type: 

Miscellaneous 

Collaboration 

An  active  data  sharing  process  for  both 
parties  involved,  not  passive.  Examples 
include:  interagency  communication 
during  crisis,  pilot/co-pilot 
communication,  and  BiLAT 
negotiations. 

Task  Type: 

Miscellaneous 

Computer  Skills  (non¬ 
military) 

Using  non-military  computer  software, 
in  a  business  or  academic  setting. 
Typically  the  tasks  are  procedural  in 
nature.  Examples  include:  word 
processing,  computer-aided  drawing 
program,  and  web-based  literature 
search  tools. 

A9 


Category 

Attribute  Name 

Attribute  Definition 

Task  Type: 

Miscellaneous 

Equipment  Diagnostics  & 
Maintenance 

Tasks  and  skills  needed  to  keep 
mechanical  or  electronic  systems  in 
good  working  order.  If  a  system 
breaks,  then  having  the  ability  to  locate 
and  correct  faults  in  mechanical  or 
electronic  system.  Examples  include: 
automotive  maintenance,  equipment 
troubleshooting  and  fault  diagnosis  of 
complex  systems. 

Task  Type: 

Miscellaneous 

Equipment  Usage 

The  use  of  job  specific  mechanical  or 
electronic  equipment.  Examples 
include:  rifle  marksmanship  and 
procedures,  using  a  digital  watch,  using 
a  voltmeter  and  using  medical 
equipment. 

Task  Type: 

Miscellaneous 

Laboratory  Task 

A  task  that  does  not  directly  translate 
to  a  real-world  task  domain  and  which 
is  conducted  in  a  laboratory  setting. 
These  tasks  are  generally  used  to 
measure  training  with  respects  to 
specific  task  categories,  for  example 
cognitive-complex  or  psychomotor. 
Examples  include:  lever-positioning, 
stimulus-response  pairing,  rules  from 
examples  and  serial  motor  task. 

Task  Type: 

Miscellaneous 

Managerial 

Making  business  decisions  and  lead 
subordinates  within  a  company. 
Managerial  skills  include  interpersonal 
skills  (relationship  building, 
negotiation),  technical  skills 
(knowledge  of  trade),  and  conceptual 
skills  (idea  development  and  strategy 
implementation).  Examples  include: 
delegating  authority,  implementing 
executive  decisions  and  labor  relations. 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Type: 

Miscellaneous 

Medical  Domain 

A  wide  range  of  knowledge  and  skills 
required  to  operate  within  in  the 
various  branches  of  the  medical 
industry.  Examples  include:  medical 
diagnosis  and  treatment  procedures, 
nursing,  surgery  and  medical  related 
college  courses. 

Task  Type: 

Miscellaneous 

Science,  Math  or 
Programming  Skills 

A  college  or  high  school  course 
teaching  subject  matter  that  requires 
quantitative  and  logical  reasoning 
skills;  including  all  branches  of 
science,  mathematics  and  computer 
programming.  Examples  include: 
chemistry,  geometry,  accounting  and 
artificial  intelligence  design. 

Task  Type: 

Miscellaneous 

Situation  Assessment 

The  evaluation,  interpretation  and 
awareness  of  data  gathered  from  one  or 
more  sources.  Examples  include: 
simulated  radar  tracking  and  decision 
making  task,  air  traffic  control, 
unmanned  vehicle  asset  allocation  and 
operation,  and  using  digital  command 
and  control  systems  to  identify  and 
communicate  potential  threats. 

Task  Type: 

Miscellaneous 

Sports 

An  activity  involving  physical  exertion 
and  skill  in  which  an  individual  or 
team  competes  against  another  or 
others. 

Task  Type: 

Miscellaneous 

Target  Identification 

Classifying  a  target  based  on  the 
examination  of  its  characteristics.  For 
example:  identifying  friend  and  foe 
spacecraft  in  the  software  Space 

Fortress,  discriminating  between 
surface-to-air  missiles,  Anti-Air 

Warfare  Coordinator  (AAWC)  tasks 
and  air  traffic  controller  tasks. 
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Category 

Attribute  Name 

Attribute  Definition 

Task  Type: 

Miscellaneous 

Tracking 

Viewing  and  keeping  a  moving  target 
in  sight.  Examples  include:  target 
tracking,  using  a  radar  tracking  system 
and  compensatory  tracking. 

Task  Type: 

Miscellaneous 

Vehicle  Control 

Tasks  involved  in  navigating,  guiding 
and  maneuvering  an  air,  ground  or 
water  vehicle.  Examples  include:  flight 
control,  submarine  navigation,  driving 
a  car  and  operating  an  unmanned  aerial 
vehicle. 

Individual  Factors 

Ability 

The  report  presents  a  relationship 
between  training  effectiveness  and 
some  measured  aspect  of  the  trainee’s 
task-related  ability,  skill  or  aptitude. 
Examples  include:  ASVAB,  verbal 
comprehension,  numerical  reasoning 
ability  and  mechanical  ability. 

Individual  Factors 

Age 

The  report  presents  a  relationship 
between  training  effectiveness  and 
some  measured  aspect  of  the  trainee’s 
age. 

Individual  Factors 

Education  Level 

The  report  presents  a  relationship 
between  training  effectiveness  and 
some  measured  aspect  of  the  trainee’s 
level  of  education.  Examples  include: 
undergraduate,  masters,  and  high 
school  diploma. 

Individual  Factors 

Experience 

The  report  presents  a  relationship 
between  training  effectiveness  and 
some  measured  aspect  of  the  trainee’s 
familiarity  with  the  task  content.  For 
example:  prior  knowledge  of  task  and 
military  experience. 

Individual  Factors 

Motivation 

The  report  presents  a  relationship 
between  training  effectiveness  and 
some  measured  aspect  of  the  trainee’s 
incentive  to  learn  or  be  trained. 
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Category 

Attribute  Name 

Attribute  Definition 

Outcome  Measure 

Learner  Reaction 

The  report  presents  a  measure  of 
trainee  affect  or  perceptions  before, 
during,  or  after  training  administration. 

Outcome  Measure 

Learning 

The  report  presents  a  measure  of  target 
task  or  skill  performance  evaluated  in 
terms  of  either  the  rate  of  learning  the 
target  task  during  training,  a 
declarative  or  conceptual  knowledge 
test,  or  a  test  in  which  an  actual  task  is 
performed  and  evaluated  that  utilizes 
the  skills  trained. 

Outcome  Measure 

Retention 

The  report  presents  a  measure  of  target 
task  performance  following  a  period  of 
time  after  training  administration  to 
assess  degradation  of  performance. 

Outcome  Measure 

Transfer 

The  report  presents  a  measure  of  what 
was  learned  in  training  in  a  test  with  a 
task  or  problem  somewhat  different, 
but  similar  to  that  used  in  training. 

Outcome  Measure 

Transfer- Job 

The  report  presents  a  measure  of 
performance  of  the  trained  task/skill  in 
a  work  environment. 

Meta-analysis 

Training  Difficulty 

Report  results  are  included  in  training 
difficulty  meta-analysis. 

Sample 

Academic 

Study  sample  is  undergraduate  or  high 
school  students. 

Sample 

Military 

Study  sample  is  military  personnel. 

Sample 

Work 

Study  sample  is  people  in  the 
workforce. 

Delivery  Method 

Classroom 

Delivery  Method 

Comparing  Delivery 
Methods 

This  document  compares  different 
delivery  methods. 
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Category 

Attribute  Name 

Attribute  Definition 

Delivery  Method 

Computer-based 

The  primary  delivery  environment  is  a 
computer-based  presentation  of  the 
training  material.  The  computer-based 
training  environment  can  be  online 
(web-based),  on  a  local  network,  or  on 
a  single  computer.  The  training 
methods  are  implemented  within  the 
computer-based  environment.  Includes 
CAI,  CBT,  and  many  ITS  systems. 

Delivery  Method 

Field/Real  System 

The  primary  delivery  environment  is  a 
field  exercise  with  real  equipment  or 
training  using  the  actual  operational 
system  or  device.  This  includes 
training  to  use  computer  applications, 
sports,  real  science  labs  etc. 

Delivery  Method 

Gaming 

Interactive  simulation  with  competition 
&  scoring  as  well  as  constraints, 
privileges  and  penalties. 

Delivery  Method 

Simulation-based 

The  primary  delivery  environment  is 
an  interactive  synthetic  environment 
that  approximates  the  real  world 
environment  or  equipment  in  which  the 
target  task  is  to  be  performed.  It  can 
include  desktop  system  simulations, 
equipment  simulators,  virtual 
environments,  and  networked 
environments.  It  also  includes 
simulation  gaming  environments.  A 
key  difference  from  computer-based  is 
that  the  trainee  manipulates  variables 
in  the  environment  to  change  system 
states  and  outcomes. 

Delivery  Method 

Web-based 

Instruction  which  is  delivered  via  a 
computer  using  the  Internet,  enabling 
instant  updating,  distribution,  and 
sharing  of  infonnation.  Gives  learners 
the  ability  to  communicate 
interactively  with  others  online  and 
access  to  other  information  and  media 
via  the  web. 
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